2024-02-10T08:01:42.462 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-10T08:01:42.462 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298 --description rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} --owner scheduled_yuriw@teuthology --verbose --name yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/orig.config.yaml 2024-02-10T08:01:42.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T08:01:42.571 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298 branch: wip-yuri10-testing-2024-02-08-0854-pacific description: rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} email: null first_in_suite: false job_id: '7555298' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri10-testing-2024-02-08-0854-pacific ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(MON_DOWN\) - missing hit_sets - do not have an application enabled sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 extra_packages: - cephadm kernel: hwe: true workunit: branch: wip-tracker-64343-pacific sha1: 57bd6abdec7bb457ae7999d9c96682e9ac678e27 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 964 sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-tracker-64343-pacific suite_path: /home/teuthworker/src/github.com_ljflores_ceph_57bd6abdec7bb457ae7999d9c96682e9ac678e27/qa suite_relpath: qa suite_repo: https://github.com/ljflores/ceph.git suite_sha1: 57bd6abdec7bb457ae7999d9c96682e9ac678e27 targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-09_21:34:44 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-10T08:01:42.571 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ljflores_ceph_57bd6abdec7bb457ae7999d9c96682e9ac678e27/qa; will attempt to use it 2024-02-10T08:01:42.572 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ljflores_ceph_57bd6abdec7bb457ae7999d9c96682e9ac678e27/qa/tasks 2024-02-10T08:01:42.572 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-10T08:01:42.574 INFO:teuthology.task.internal:Checking packages... 2024-02-10T08:01:42.595 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '0e714d9a4bd2a821113e6318adb87bd06cf81ec1' 2024-02-10T08:01:42.595 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-10T08:01:42.595 INFO:teuthology.packaging:ref: None 2024-02-10T08:01:42.595 INFO:teuthology.packaging:tag: None 2024-02-10T08:01:42.596 INFO:teuthology.packaging:branch: wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-10T08:01:42.596 INFO:teuthology.packaging:sha1: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:01:42.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri10-testing-2024-02-08-0854-pacific 2024-02-10T08:01:42.749 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-521-g0e714d9a-1focal 2024-02-10T08:01:42.750 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-10T08:01:42.761 INFO:teuthology.task.internal:no buildpackages task found 2024-02-10T08:01:42.761 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-10T08:01:42.788 INFO:teuthology.task.internal:Saving configuration 2024-02-10T08:01:42.799 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-10T08:01:42.809 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T08:01:42.835 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:54:47.659329', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-02-10T08:01:42.856 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:54:47.660806', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-02-10T08:01:42.856 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-10T08:01:42.867 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-10T08:01:42.867 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-10T08:01:42.867 INFO:teuthology.run_tasks:Running task console_log... 2024-02-10T08:01:42.942 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd51e6d70d0>, signals=[15]) 2024-02-10T08:01:42.943 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-10T08:01:42.953 INFO:teuthology.task.internal:Opening connections... 2024-02-10T08:01:42.953 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-02-10T08:01:42.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:01:43.048 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-02-10T08:01:43.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:01:43.143 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-10T08:01:43.154 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-02-10T08:01:43.161 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-02-10T08:01:43.161 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="Ubuntu" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:ID=ubuntu 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE=debian 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="20.04" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://www.ubuntu.com/" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-02-10T08:01:43.209 INFO:teuthology.orchestra.run.smithi067.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-02-10T08:01:43.210 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_CODENAME=focal 2024-02-10T08:01:43.210 INFO:teuthology.orchestra.run.smithi067.stdout:UBUNTU_CODENAME=focal 2024-02-10T08:01:43.210 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-02-10T08:01:43.233 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-02-10T08:01:43.240 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-02-10T08:01:43.240 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-02-10T08:01:43.290 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Ubuntu" 2024-02-10T08:01:43.290 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-02-10T08:01:43.290 INFO:teuthology.orchestra.run.smithi134.stdout:ID=ubuntu 2024-02-10T08:01:43.290 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE=debian 2024-02-10T08:01:43.290 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-02-10T08:01:43.290 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="20.04" 2024-02-10T08:01:43.291 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.ubuntu.com/" 2024-02-10T08:01:43.291 INFO:teuthology.orchestra.run.smithi134.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-02-10T08:01:43.291 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-02-10T08:01:43.291 INFO:teuthology.orchestra.run.smithi134.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-02-10T08:01:43.291 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_CODENAME=focal 2024-02-10T08:01:43.291 INFO:teuthology.orchestra.run.smithi134.stdout:UBUNTU_CODENAME=focal 2024-02-10T08:01:43.291 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-02-10T08:01:43.316 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-10T08:01:43.327 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-10T08:01:43.341 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-10T08:01:43.341 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-02-10T08:01:43.343 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-02-10T08:01:43.349 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-10T08:01:43.360 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-10T08:01:43.360 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-02-10T08:01:43.389 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-02-10T08:01:43.477 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-10T08:01:43.538 INFO:teuthology.run_tasks:Running task kernel... 2024-02-10T08:01:43.557 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-10T08:01:43.557 INFO:teuthology.task.kernel:normalize config orig: {'hwe': True} 2024-02-10T08:01:43.557 DEBUG:teuthology.task.kernel:normalized overrides {'mon.a': {'hwe': True}, 'mon.c': {'hwe': True}, 'mgr.y': {'hwe': True}, 'osd.0': {'hwe': True}, 'osd.1': {'hwe': True}, 'osd.2': {'hwe': True}, 'osd.3': {'hwe': True}, 'client.0': {'hwe': True}, 'ceph.rgw.foo.a': {'hwe': True}, 'node-exporter.a': {'hwe': True}, 'alertmanager.a': {'hwe': True}, 'mon.b': {'hwe': True}, 'mgr.x': {'hwe': True}, 'osd.4': {'hwe': True}, 'osd.5': {'hwe': True}, 'osd.6': {'hwe': True}, 'osd.7': {'hwe': True}, 'client.1': {'hwe': True}, 'prometheus.a': {'hwe': True}, 'grafana.a': {'hwe': True}, 'node-exporter.b': {'hwe': True}, 'ceph.iscsi.iscsi.a': {'hwe': True}} 2024-02-10T08:01:43.557 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro', 'hwe': True}, 'mon.b': {'kdb': True, 'sha1': 'distro', 'hwe': True}}, timeout 300 2024-02-10T08:01:43.557 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T08:01:43.558 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T08:01:43.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:01:43.563 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-02-10T08:01:43.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:01:43.564 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-02-10T08:01:43.613 INFO:teuthology.orchestra.run.smithi067.stdout:5.4.0-124-generic 2024-02-10T08:01:43.613 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.4.0-124-generic 2024-02-10T08:01:43.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo apt-get clean 2024-02-10T08:01:43.614 INFO:teuthology.orchestra.run.smithi134.stdout:5.4.0-124-generic 2024-02-10T08:01:43.614 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.4.0-124-generic 2024-02-10T08:01:43.614 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get clean 2024-02-10T08:01:43.704 DEBUG:teuthology.orchestra.run.smithi067:> sudo apt-get update 2024-02-10T08:01:43.746 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-02-10T08:01:43.875 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-02-10T08:01:43.888 INFO:teuthology.orchestra.run.smithi067.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-02-10T08:01:43.913 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-02-10T08:01:43.932 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2024-02-10T08:01:43.962 INFO:teuthology.orchestra.run.smithi067.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-02-10T08:01:43.980 INFO:teuthology.orchestra.run.smithi067.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-02-10T08:01:44.009 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2024-02-10T08:01:44.019 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2024-02-10T08:01:44.254 INFO:teuthology.orchestra.run.smithi067.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,090 kB] 2024-02-10T08:01:44.292 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [931 kB] 2024-02-10T08:01:44.392 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,090 kB] 2024-02-10T08:01:44.397 INFO:teuthology.orchestra.run.smithi067.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [931 kB] 2024-02-10T08:01:44.411 INFO:teuthology.orchestra.run.smithi067.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [496 kB] 2024-02-10T08:01:44.415 INFO:teuthology.orchestra.run.smithi067.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-02-10T08:01:44.415 INFO:teuthology.orchestra.run.smithi067.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [36.8 kB] 2024-02-10T08:01:44.415 INFO:teuthology.orchestra.run.smithi067.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,665 kB] 2024-02-10T08:01:44.452 INFO:teuthology.orchestra.run.smithi067.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [371 kB] 2024-02-10T08:01:44.455 INFO:teuthology.orchestra.run.smithi067.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-02-10T08:01:44.455 INFO:teuthology.orchestra.run.smithi067.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [772 kB] 2024-02-10T08:01:44.463 INFO:teuthology.orchestra.run.smithi067.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,163 kB] 2024-02-10T08:01:44.478 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [496 kB] 2024-02-10T08:01:44.478 INFO:teuthology.orchestra.run.smithi067.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [279 kB] 2024-02-10T08:01:44.485 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-02-10T08:01:44.485 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [36.8 kB] 2024-02-10T08:01:44.486 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,665 kB] 2024-02-10T08:01:44.486 INFO:teuthology.orchestra.run.smithi067.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-02-10T08:01:44.495 INFO:teuthology.orchestra.run.smithi067.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2024-02-10T08:01:44.495 INFO:teuthology.orchestra.run.smithi067.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.1 kB] 2024-02-10T08:01:44.495 INFO:teuthology.orchestra.run.smithi067.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,768 B] 2024-02-10T08:01:44.522 INFO:teuthology.orchestra.run.smithi067.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-02-10T08:01:44.523 INFO:teuthology.orchestra.run.smithi067.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-02-10T08:01:44.523 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [371 kB] 2024-02-10T08:01:44.524 INFO:teuthology.orchestra.run.smithi067.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-02-10T08:01:44.525 INFO:teuthology.orchestra.run.smithi067.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-02-10T08:01:44.525 INFO:teuthology.orchestra.run.smithi067.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-02-10T08:01:44.526 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-02-10T08:01:44.526 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [772 kB] 2024-02-10T08:01:44.530 INFO:teuthology.orchestra.run.smithi067.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-02-10T08:01:44.531 INFO:teuthology.orchestra.run.smithi067.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-02-10T08:01:44.533 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,163 kB] 2024-02-10T08:01:44.544 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [279 kB] 2024-02-10T08:01:44.546 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-02-10T08:01:44.547 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2024-02-10T08:01:44.547 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [26.1 kB] 2024-02-10T08:01:44.547 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,768 B] 2024-02-10T08:01:44.563 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-02-10T08:01:44.563 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-02-10T08:01:44.564 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-02-10T08:01:44.564 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-02-10T08:01:44.577 INFO:teuthology.orchestra.run.smithi067.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,710 kB] 2024-02-10T08:01:44.583 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-02-10T08:01:44.584 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-02-10T08:01:44.584 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-02-10T08:01:44.605 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [706 kB] 2024-02-10T08:01:44.941 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,710 kB] 2024-02-10T08:01:45.063 INFO:teuthology.orchestra.run.smithi067.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [706 kB] 2024-02-10T08:01:45.077 INFO:teuthology.orchestra.run.smithi067.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [413 kB] 2024-02-10T08:01:45.086 INFO:teuthology.orchestra.run.smithi067.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-02-10T08:01:45.086 INFO:teuthology.orchestra.run.smithi067.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [35.5 kB] 2024-02-10T08:01:45.087 INFO:teuthology.orchestra.run.smithi067.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,547 kB] 2024-02-10T08:01:45.130 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [413 kB] 2024-02-10T08:01:45.145 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-02-10T08:01:45.146 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [35.5 kB] 2024-02-10T08:01:45.146 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,547 kB] 2024-02-10T08:01:45.165 INFO:teuthology.orchestra.run.smithi067.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [355 kB] 2024-02-10T08:01:45.172 INFO:teuthology.orchestra.run.smithi067.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-02-10T08:01:45.173 INFO:teuthology.orchestra.run.smithi067.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [645 kB] 2024-02-10T08:01:45.186 INFO:teuthology.orchestra.run.smithi067.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [937 kB] 2024-02-10T08:01:45.220 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [355 kB] 2024-02-10T08:01:45.226 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-02-10T08:01:45.226 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [937 kB] 2024-02-10T08:01:45.265 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [645 kB] 2024-02-10T08:01:45.368 INFO:teuthology.orchestra.run.smithi067.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [197 kB] 2024-02-10T08:01:45.439 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [197 kB] 2024-02-10T08:01:45.791 INFO:teuthology.orchestra.run.smithi067.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-02-10T08:01:45.797 INFO:teuthology.orchestra.run.smithi067.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,188 B] 2024-02-10T08:01:45.799 INFO:teuthology.orchestra.run.smithi067.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.9 kB] 2024-02-10T08:01:45.806 INFO:teuthology.orchestra.run.smithi067.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,796 B] 2024-02-10T08:01:45.808 INFO:teuthology.orchestra.run.smithi067.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-02-10T08:01:45.868 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-02-10T08:01:45.873 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,188 B] 2024-02-10T08:01:45.875 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.9 kB] 2024-02-10T08:01:45.882 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,796 B] 2024-02-10T08:01:45.885 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-02-10T08:01:48.502 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 19.0 MB in 3s (7,032 kB/s) 2024-02-10T08:01:48.668 INFO:teuthology.orchestra.run.smithi067.stdout:Fetched 19.0 MB in 3s (6,712 kB/s) 2024-02-10T08:01:49.877 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-02-10T08:01:49.904 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-02-10T08:01:49.978 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-02-10T08:01:50.160 INFO:teuthology.orchestra.run.smithi067.stdout:Reading package lists... 2024-02-10T08:01:50.184 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-02-10T08:01:50.185 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-02-10T08:01:50.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-02-10T08:01:50.267 INFO:teuthology.orchestra.run.smithi067.stdout:Reading package lists... 2024-02-10T08:01:50.359 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-02-10T08:01:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-02-10T08:01:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: libusbmuxd6 linux-image-5.15.0-94-generic linux-modules-5.15.0-94-generic 2024-02-10T08:01:50.359 INFO:teuthology.orchestra.run.smithi134.stdout: linux-modules-extra-5.15.0-94-generic thermald upower usbmuxd 2024-02-10T08:01:50.360 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-02-10T08:01:50.360 INFO:teuthology.orchestra.run.smithi134.stdout: libusbmuxd-tools fdutils linux-doc | linux-hwe-5.15-source-5.15.0 2024-02-10T08:01:50.360 INFO:teuthology.orchestra.run.smithi134.stdout: linux-hwe-5.15-tools linux-headers-5.15.0-94-generic 2024-02-10T08:01:50.407 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-02-10T08:01:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-02-10T08:01:50.407 INFO:teuthology.orchestra.run.smithi134.stdout: libusbmuxd6 linux-image-5.15.0-94-generic linux-image-generic-hwe-20.04 2024-02-10T08:01:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: linux-modules-5.15.0-94-generic linux-modules-extra-5.15.0-94-generic 2024-02-10T08:01:50.408 INFO:teuthology.orchestra.run.smithi134.stdout: thermald upower usbmuxd 2024-02-10T08:01:50.476 INFO:teuthology.orchestra.run.smithi067.stdout:Building dependency tree... 2024-02-10T08:01:50.476 INFO:teuthology.orchestra.run.smithi067.stdout:Reading state information... 2024-02-10T08:01:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:0 upgraded, 13 newly installed, 0 to remove and 287 not upgraded. 2024-02-10T08:01:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 90.9 MB of archives. 2024-02-10T08:01:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 497 MB of additional disk space will be used. 2024-02-10T08:01:50.595 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdbus-glib-1-2 amd64 0.110-5fakssync1 [59.1 kB] 2024-02-10T08:01:50.675 INFO:teuthology.orchestra.run.smithi067.stdout:The following additional packages will be installed: 2024-02-10T08:01:50.675 INFO:teuthology.orchestra.run.smithi067.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-02-10T08:01:50.675 INFO:teuthology.orchestra.run.smithi067.stdout: libusbmuxd6 linux-image-5.15.0-94-generic linux-modules-5.15.0-94-generic 2024-02-10T08:01:50.675 INFO:teuthology.orchestra.run.smithi067.stdout: linux-modules-extra-5.15.0-94-generic thermald upower usbmuxd 2024-02-10T08:01:50.676 INFO:teuthology.orchestra.run.smithi067.stdout:Suggested packages: 2024-02-10T08:01:50.676 INFO:teuthology.orchestra.run.smithi067.stdout: libusbmuxd-tools fdutils linux-doc | linux-hwe-5.15-source-5.15.0 2024-02-10T08:01:50.676 INFO:teuthology.orchestra.run.smithi067.stdout: linux-hwe-5.15-tools linux-headers-5.15.0-94-generic 2024-02-10T08:01:50.725 INFO:teuthology.orchestra.run.smithi067.stdout:The following NEW packages will be installed: 2024-02-10T08:01:50.725 INFO:teuthology.orchestra.run.smithi067.stdout: libdbus-glib-1-2 libevdev2 libimobiledevice6 libplist3 libupower-glib3 2024-02-10T08:01:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: libusbmuxd6 linux-image-5.15.0-94-generic linux-image-generic-hwe-20.04 2024-02-10T08:01:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: linux-modules-5.15.0-94-generic linux-modules-extra-5.15.0-94-generic 2024-02-10T08:01:50.726 INFO:teuthology.orchestra.run.smithi067.stdout: thermald upower usbmuxd 2024-02-10T08:01:50.789 INFO:teuthology.orchestra.run.smithi067.stdout:0 upgraded, 13 newly installed, 0 to remove and 287 not upgraded. 2024-02-10T08:01:50.789 INFO:teuthology.orchestra.run.smithi067.stdout:Need to get 90.9 MB of archives. 2024-02-10T08:01:50.790 INFO:teuthology.orchestra.run.smithi067.stdout:After this operation, 497 MB of additional disk space will be used. 2024-02-10T08:01:50.790 INFO:teuthology.orchestra.run.smithi067.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libdbus-glib-1-2 amd64 0.110-5fakssync1 [59.1 kB] 2024-02-10T08:01:50.859 INFO:teuthology.orchestra.run.smithi067.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libplist3 amd64 2.1.0-4build2 [31.6 kB] 2024-02-10T08:01:50.866 INFO:teuthology.orchestra.run.smithi067.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libusbmuxd6 amd64 2.0.1-2 [19.1 kB] 2024-02-10T08:01:50.870 INFO:teuthology.orchestra.run.smithi067.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libimobiledevice6 amd64 1.2.1~git20191129.9f79242-1build1 [65.2 kB] 2024-02-10T08:01:50.879 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libplist3 amd64 2.1.0-4build2 [31.6 kB] 2024-02-10T08:01:50.889 INFO:teuthology.orchestra.run.smithi067.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libupower-glib3 amd64 0.99.11-1build2 [43.2 kB] 2024-02-10T08:01:50.898 INFO:teuthology.orchestra.run.smithi067.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.15.0-94-generic amd64 5.15.0-94.104~20.04.1 [21.0 MB] 2024-02-10T08:01:50.906 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libusbmuxd6 amd64 2.0.1-2 [19.1 kB] 2024-02-10T08:01:50.924 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 libimobiledevice6 amd64 1.2.1~git20191129.9f79242-1build1 [65.2 kB] 2024-02-10T08:01:50.995 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 libupower-glib3 amd64 0.99.11-1build2 [43.2 kB] 2024-02-10T08:01:51.030 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.15.0-94-generic amd64 5.15.0-94.104~20.04.1 [21.0 MB] 2024-02-10T08:01:51.208 INFO:teuthology.orchestra.run.smithi067.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.15.0-94-generic amd64 5.15.0-94.104~20.04.1 [11.4 MB] 2024-02-10T08:01:51.420 INFO:teuthology.orchestra.run.smithi067.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.15.0-94-generic amd64 5.15.0-94.104~20.04.1 [57.9 MB] 2024-02-10T08:01:51.954 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.15.0-94-generic amd64 5.15.0-94.104~20.04.1 [11.4 MB] 2024-02-10T08:01:52.223 INFO:teuthology.orchestra.run.smithi067.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic-hwe-20.04 amd64 5.15.0.94.104~20.04.50 [2,596 B] 2024-02-10T08:01:52.244 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.15.0-94-generic amd64 5.15.0-94.104~20.04.1 [57.9 MB] 2024-02-10T08:01:52.295 INFO:teuthology.orchestra.run.smithi067.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libevdev2 amd64 1.9.0+dfsg-1ubuntu0.2 [31.6 kB] 2024-02-10T08:01:52.296 INFO:teuthology.orchestra.run.smithi067.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 thermald amd64 1.9.1-1ubuntu0.6 [233 kB] 2024-02-10T08:01:52.302 INFO:teuthology.orchestra.run.smithi067.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 upower amd64 0.99.11-1build2 [104 kB] 2024-02-10T08:01:52.304 INFO:teuthology.orchestra.run.smithi067.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 usbmuxd amd64 1.1.1~git20191130.9af2b12-1 [38.4 kB] 2024-02-10T08:01:52.951 INFO:teuthology.orchestra.run.smithi067.stdout:Fetched 90.9 MB in 2s (57.9 MB/s) 2024-02-10T08:01:53.169 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libdbus-glib-1-2:amd64. 2024-02-10T08:01:54.122 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic-hwe-20.04 amd64 5.15.0.94.104~20.04.50 [2,596 B] 2024-02-10T08:01:54.200 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libevdev2 amd64 1.9.0+dfsg-1ubuntu0.2 [31.6 kB] 2024-02-10T08:01:54.201 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 thermald amd64 1.9.1-1ubuntu0.6 [233 kB] 2024-02-10T08:01:54.370 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 upower amd64 0.99.11-1build2 [104 kB] 2024-02-10T08:01:54.724 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 usbmuxd amd64 1.1.1~git20191130.9af2b12-1 [38.4 kB] 2024-02-10T08:01:54.906 INFO:teuthology.orchestra.run.smithi067.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-02-10T08:01:54.912 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../00-libdbus-glib-1-2_0.110-5fakssync1_amd64.deb ... 2024-02-10T08:01:55.050 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-02-10T08:01:55.403 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libplist3:amd64. 2024-02-10T08:01:55.415 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 90.9 MB in 4s (21.0 MB/s) 2024-02-10T08:01:55.418 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../01-libplist3_2.1.0-4build2_amd64.deb ... 2024-02-10T08:01:55.461 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libplist3:amd64 (2.1.0-4build2) ... 2024-02-10T08:01:55.622 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libdbus-glib-1-2:amd64. 2024-02-10T08:01:55.822 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libusbmuxd6:amd64. 2024-02-10T08:01:55.836 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../02-libusbmuxd6_2.0.1-2_amd64.deb ... 2024-02-10T08:01:55.880 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libusbmuxd6:amd64 (2.0.1-2) ... 2024-02-10T08:01:56.142 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libimobiledevice6:amd64. 2024-02-10T08:01:56.157 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../03-libimobiledevice6_1.2.1~git20191129.9f79242-1build1_amd64.deb ... 2024-02-10T08:01:56.217 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-02-10T08:01:56.561 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libupower-glib3:amd64. 2024-02-10T08:01:56.575 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../04-libupower-glib3_0.99.11-1build2_amd64.deb ... 2024-02-10T08:01:56.619 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libupower-glib3:amd64 (0.99.11-1build2) ... 2024-02-10T08:01:56.964 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package linux-modules-5.15.0-94-generic. 2024-02-10T08:01:56.980 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../05-linux-modules-5.15.0-94-generic_5.15.0-94.104~20.04.1_amd64.deb ... 2024-02-10T08:01:57.022 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking linux-modules-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:01:57.319 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-02-10T08:01:57.324 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../00-libdbus-glib-1-2_0.110-5fakssync1_amd64.deb ... 2024-02-10T08:01:57.444 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-02-10T08:01:57.831 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libplist3:amd64. 2024-02-10T08:01:57.845 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../01-libplist3_2.1.0-4build2_amd64.deb ... 2024-02-10T08:01:57.889 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libplist3:amd64 (2.1.0-4build2) ... 2024-02-10T08:01:58.209 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libusbmuxd6:amd64. 2024-02-10T08:01:58.222 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../02-libusbmuxd6_2.0.1-2_amd64.deb ... 2024-02-10T08:01:58.259 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libusbmuxd6:amd64 (2.0.1-2) ... 2024-02-10T08:01:58.578 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libimobiledevice6:amd64. 2024-02-10T08:01:58.593 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../03-libimobiledevice6_1.2.1~git20191129.9f79242-1build1_amd64.deb ... 2024-02-10T08:01:58.636 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-02-10T08:01:58.965 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libupower-glib3:amd64. 2024-02-10T08:01:58.978 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../04-libupower-glib3_0.99.11-1build2_amd64.deb ... 2024-02-10T08:01:59.023 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libupower-glib3:amd64 (0.99.11-1build2) ... 2024-02-10T08:01:59.368 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-modules-5.15.0-94-generic. 2024-02-10T08:01:59.381 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../05-linux-modules-5.15.0-94-generic_5.15.0-94.104~20.04.1_amd64.deb ... 2024-02-10T08:01:59.426 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-modules-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:01:59.515 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package linux-image-5.15.0-94-generic. 2024-02-10T08:01:59.529 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../06-linux-image-5.15.0-94-generic_5.15.0-94.104~20.04.1_amd64.deb ... 2024-02-10T08:01:59.640 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking linux-image-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:00.185 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-94-generic. 2024-02-10T08:02:00.198 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../07-linux-modules-extra-5.15.0-94-generic_5.15.0-94.104~20.04.1_amd64.deb ... 2024-02-10T08:02:00.235 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking linux-modules-extra-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:01.953 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-image-5.15.0-94-generic. 2024-02-10T08:02:01.966 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../06-linux-image-5.15.0-94-generic_5.15.0-94.104~20.04.1_amd64.deb ... 2024-02-10T08:02:02.174 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-image-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:02.739 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-94-generic. 2024-02-10T08:02:02.753 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../07-linux-modules-extra-5.15.0-94-generic_5.15.0-94.104~20.04.1_amd64.deb ... 2024-02-10T08:02:02.797 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-modules-extra-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:07.402 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package linux-image-generic-hwe-20.04. 2024-02-10T08:02:07.421 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../08-linux-image-generic-hwe-20.04_5.15.0.94.104~20.04.50_amd64.deb ... 2024-02-10T08:02:07.485 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking linux-image-generic-hwe-20.04 (5.15.0.94.104~20.04.50) ... 2024-02-10T08:02:07.813 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libevdev2:amd64. 2024-02-10T08:02:07.828 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../09-libevdev2_1.9.0+dfsg-1ubuntu0.2_amd64.deb ... 2024-02-10T08:02:07.871 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-02-10T08:02:08.257 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package thermald. 2024-02-10T08:02:08.272 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../10-thermald_1.9.1-1ubuntu0.6_amd64.deb ... 2024-02-10T08:02:08.315 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking thermald (1.9.1-1ubuntu0.6) ... 2024-02-10T08:02:08.879 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package upower. 2024-02-10T08:02:08.893 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../11-upower_0.99.11-1build2_amd64.deb ... 2024-02-10T08:02:08.957 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking upower (0.99.11-1build2) ... 2024-02-10T08:02:09.375 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package usbmuxd. 2024-02-10T08:02:09.388 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../12-usbmuxd_1.1.1~git20191130.9af2b12-1_amd64.deb ... 2024-02-10T08:02:09.433 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-02-10T08:02:09.623 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-image-generic-hwe-20.04. 2024-02-10T08:02:09.637 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../08-linux-image-generic-hwe-20.04_5.15.0.94.104~20.04.50_amd64.deb ... 2024-02-10T08:02:09.681 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-image-generic-hwe-20.04 (5.15.0.94.104~20.04.50) ... 2024-02-10T08:02:09.741 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libplist3:amd64 (2.1.0-4build2) ... 2024-02-10T08:02:09.863 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-02-10T08:02:09.989 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libusbmuxd6:amd64 (2.0.1-2) ... 2024-02-10T08:02:10.008 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libevdev2:amd64. 2024-02-10T08:02:10.022 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../09-libevdev2_1.9.0+dfsg-1ubuntu0.2_amd64.deb ... 2024-02-10T08:02:10.066 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-02-10T08:02:10.115 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libupower-glib3:amd64 (0.99.11-1build2) ... 2024-02-10T08:02:10.274 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-02-10T08:02:10.400 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-02-10T08:02:10.445 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package thermald. 2024-02-10T08:02:10.458 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../10-thermald_1.9.1-1ubuntu0.6_amd64.deb ... 2024-02-10T08:02:10.502 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking thermald (1.9.1-1ubuntu0.6) ... 2024-02-10T08:02:10.526 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up upower (0.99.11-1build2) ... 2024-02-10T08:02:11.041 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package upower. 2024-02-10T08:02:11.055 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../11-upower_0.99.11-1build2_amd64.deb ... 2024-02-10T08:02:11.093 INFO:teuthology.orchestra.run.smithi067.stdout:upower.service is a disabled or a static unit, not starting it. 2024-02-10T08:02:11.114 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking upower (0.99.11-1build2) ... 2024-02-10T08:02:11.147 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-02-10T08:02:11.325 INFO:teuthology.orchestra.run.smithi067.stdout:Warning: The home dir /var/lib/usbmux you specified can't be accessed: No such file or directory 2024-02-10T08:02:11.338 INFO:teuthology.orchestra.run.smithi067.stdout:Adding system user `usbmux' (UID 114) ... 2024-02-10T08:02:11.338 INFO:teuthology.orchestra.run.smithi067.stdout:Adding new user `usbmux' (UID 114) with group `plugdev' ... 2024-02-10T08:02:11.512 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package usbmuxd. 2024-02-10T08:02:11.526 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../12-usbmuxd_1.1.1~git20191130.9af2b12-1_amd64.deb ... 2024-02-10T08:02:11.561 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-02-10T08:02:11.867 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libplist3:amd64 (2.1.0-4build2) ... 2024-02-10T08:02:11.881 INFO:teuthology.orchestra.run.smithi067.stdout:Not creating home directory `/var/lib/usbmux'. 2024-02-10T08:02:12.008 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libdbus-glib-1-2:amd64 (0.110-5fakssync1) ... 2024-02-10T08:02:12.113 INFO:teuthology.orchestra.run.smithi067.stdout:usbmuxd.service is a disabled or a static unit, not starting it. 2024-02-10T08:02:12.133 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libusbmuxd6:amd64 (2.0.1-2) ... 2024-02-10T08:02:12.161 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up thermald (1.9.1-1ubuntu0.6) ... 2024-02-10T08:02:12.243 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libupower-glib3:amd64 (0.99.11-1build2) ... 2024-02-10T08:02:12.369 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libimobiledevice6:amd64 (1.2.1~git20191129.9f79242-1build1) ... 2024-02-10T08:02:12.495 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libevdev2:amd64 (1.9.0+dfsg-1ubuntu0.2) ... 2024-02-10T08:02:12.621 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up upower (0.99.11-1build2) ... 2024-02-10T08:02:12.656 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.thermald.service -> /lib/systemd/system/thermald.service. 2024-02-10T08:02:12.656 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/thermald.service -> /lib/systemd/system/thermald.service. 2024-02-10T08:02:13.133 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up linux-image-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:13.170 INFO:teuthology.orchestra.run.smithi134.stdout:upower.service is a disabled or a static unit, not starting it. 2024-02-10T08:02:13.225 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up usbmuxd (1.1.1~git20191130.9af2b12-1) ... 2024-02-10T08:02:13.457 INFO:teuthology.orchestra.run.smithi134.stdout:Warning: The home dir /var/lib/usbmux you specified can't be accessed: No such file or directory 2024-02-10T08:02:13.463 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user `usbmux' (UID 114) ... 2024-02-10T08:02:13.463 INFO:teuthology.orchestra.run.smithi134.stdout:Adding new user `usbmux' (UID 114) with group `plugdev' ... 2024-02-10T08:02:13.933 INFO:teuthology.orchestra.run.smithi134.stdout:Not creating home directory `/var/lib/usbmux'. 2024-02-10T08:02:14.148 INFO:teuthology.orchestra.run.smithi134.stdout:usbmuxd.service is a disabled or a static unit, not starting it. 2024-02-10T08:02:14.188 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up thermald (1.9.1-1ubuntu0.6) ... 2024-02-10T08:02:14.486 INFO:teuthology.orchestra.run.smithi067.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-94-generic 2024-02-10T08:02:14.487 INFO:teuthology.orchestra.run.smithi067.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:14.499 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.thermald.service -> /lib/systemd/system/thermald.service. 2024-02-10T08:02:14.500 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/thermald.service -> /lib/systemd/system/thermald.service. 2024-02-10T08:02:14.642 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up linux-modules-extra-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:14.993 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-image-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:16.177 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up linux-image-generic-hwe-20.04 (5.15.0.94.104~20.04.50) ... 2024-02-10T08:02:16.288 INFO:teuthology.orchestra.run.smithi134.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-94-generic 2024-02-10T08:02:16.288 INFO:teuthology.orchestra.run.smithi134.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:16.336 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up linux-modules-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:16.444 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-modules-extra-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:17.853 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-image-generic-hwe-20.04 (5.15.0.94.104~20.04.50) ... 2024-02-10T08:02:17.972 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-02-10T08:02:18.260 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-02-10T08:02:19.014 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-modules-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:19.021 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-02-10T08:02:19.155 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for linux-image-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:19.230 INFO:teuthology.orchestra.run.smithi067.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-02-10T08:02:19.230 INFO:teuthology.orchestra.run.smithi067.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:20.390 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-02-10T08:02:20.670 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-02-10T08:02:21.431 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-02-10T08:02:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for linux-image-5.15.0-94-generic (5.15.0-94.104~20.04.1) ... 2024-02-10T08:02:21.623 INFO:teuthology.orchestra.run.smithi134.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-02-10T08:02:21.623 INFO:teuthology.orchestra.run.smithi134.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:45.187 INFO:teuthology.orchestra.run.smithi067.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-02-10T08:02:45.187 INFO:teuthology.orchestra.run.smithi067.stdout:Sourcing file `/etc/default/grub' 2024-02-10T08:02:45.202 INFO:teuthology.orchestra.run.smithi067.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-02-10T08:02:45.221 INFO:teuthology.orchestra.run.smithi067.stdout:Generating grub configuration file ... 2024-02-10T08:02:45.576 INFO:teuthology.orchestra.run.smithi067.stdout:Found linux image: /boot/vmlinuz-5.15.0-94-generic 2024-02-10T08:02:45.598 INFO:teuthology.orchestra.run.smithi067.stdout:Found initrd image: /boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:46.022 INFO:teuthology.orchestra.run.smithi067.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-02-10T08:02:46.034 INFO:teuthology.orchestra.run.smithi067.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-02-10T08:02:47.008 INFO:teuthology.orchestra.run.smithi134.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-02-10T08:02:47.009 INFO:teuthology.orchestra.run.smithi134.stdout:Sourcing file `/etc/default/grub' 2024-02-10T08:02:47.015 INFO:teuthology.orchestra.run.smithi134.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-02-10T08:02:47.030 INFO:teuthology.orchestra.run.smithi134.stdout:Generating grub configuration file ... 2024-02-10T08:02:47.385 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.15.0-94-generic 2024-02-10T08:02:47.409 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:47.756 INFO:teuthology.orchestra.run.smithi067.stdout:done 2024-02-10T08:02:47.823 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-02-10T08:02:47.834 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-02-10T08:02:48.160 DEBUG:teuthology.orchestra.run.smithi067:> dpkg -s linux-image-generic-hwe-20.04 2024-02-10T08:02:48.197 INFO:teuthology.orchestra.run.smithi067.stdout:Package: linux-image-generic-hwe-20.04 2024-02-10T08:02:48.197 INFO:teuthology.orchestra.run.smithi067.stdout:Status: install ok installed 2024-02-10T08:02:48.197 INFO:teuthology.orchestra.run.smithi067.stdout:Priority: optional 2024-02-10T08:02:48.197 INFO:teuthology.orchestra.run.smithi067.stdout:Section: kernel 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Installed-Size: 20 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Maintainer: Ubuntu Kernel Team 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: amd64 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Source: linux-meta-hwe-5.15 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Version: 5.15.0.94.104~20.04.50 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-94), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Depends: linux-image-5.15.0-94-generic, linux-modules-extra-5.15.0-94-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Recommends: thermald 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout:Description: Generic Linux kernel image 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout: This package will always depend on the latest generic kernel image 2024-02-10T08:02:48.198 INFO:teuthology.orchestra.run.smithi067.stdout: available. 2024-02-10T08:02:48.199 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.15.0-94-generic 2024-02-10T08:02:48.199 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T08:02:48.199 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T08:02:48.199 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-10T08:02:48.200 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T08:02:48.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo apt-get clean 2024-02-10T08:02:48.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo apt-get update 2024-02-10T08:02:48.465 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-02-10T08:02:48.577 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-02-10T08:02:48.659 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-02-10T08:02:48.741 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-02-10T08:02:49.573 INFO:teuthology.orchestra.run.smithi134.stdout:done 2024-02-10T08:02:49.971 DEBUG:teuthology.orchestra.run.smithi134:> dpkg -s linux-image-generic-hwe-20.04 2024-02-10T08:02:50.056 INFO:teuthology.orchestra.run.smithi134.stdout:Package: linux-image-generic-hwe-20.04 2024-02-10T08:02:50.056 INFO:teuthology.orchestra.run.smithi134.stdout:Status: install ok installed 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Priority: optional 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Section: kernel 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Installed-Size: 20 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Maintainer: Ubuntu Kernel Team 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: amd64 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Source: linux-meta-hwe-5.15 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Version: 5.15.0.94.104~20.04.50 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-94), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Depends: linux-image-5.15.0-94-generic, linux-modules-extra-5.15.0-94-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Recommends: thermald 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout:Description: Generic Linux kernel image 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout: This package will always depend on the latest generic kernel image 2024-02-10T08:02:50.057 INFO:teuthology.orchestra.run.smithi134.stdout: available. 2024-02-10T08:02:50.058 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.15.0-94-generic 2024-02-10T08:02:50.058 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T08:02:50.058 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T08:02:50.058 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-10T08:02:50.058 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T08:02:50.058 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get clean 2024-02-10T08:02:50.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-02-10T08:02:50.283 INFO:teuthology.orchestra.run.smithi067.stdout:Reading package lists... 2024-02-10T08:02:50.285 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-02-10T08:02:50.295 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-02-10T08:02:50.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-02-10T08:02:50.315 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-02-10T08:02:50.382 INFO:teuthology.orchestra.run.smithi067.stdout:Reading package lists... 2024-02-10T08:02:50.397 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-02-10T08:02:50.570 INFO:teuthology.orchestra.run.smithi067.stdout:Building dependency tree... 2024-02-10T08:02:50.571 INFO:teuthology.orchestra.run.smithi067.stdout:Reading state information... 2024-02-10T08:02:50.793 INFO:teuthology.orchestra.run.smithi067.stdout:linux-image-generic-hwe-20.04 is already the newest version (5.15.0.94.104~20.04.50). 2024-02-10T08:02:50.793 INFO:teuthology.orchestra.run.smithi067.stdout:0 upgraded, 0 newly installed, 0 to remove and 287 not upgraded. 2024-02-10T08:02:50.795 DEBUG:teuthology.orchestra.run.smithi067:> dpkg -s linux-image-generic-hwe-20.04 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Package: linux-image-generic-hwe-20.04 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Status: install ok installed 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Priority: optional 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Section: kernel 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Installed-Size: 20 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Maintainer: Ubuntu Kernel Team 2024-02-10T08:02:50.818 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: amd64 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout:Source: linux-meta-hwe-5.15 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout:Version: 5.15.0.94.104~20.04.50 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-94), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout:Depends: linux-image-5.15.0-94-generic, linux-modules-extra-5.15.0-94-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout:Recommends: thermald 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout:Description: Generic Linux kernel image 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout: This package will always depend on the latest generic kernel image 2024-02-10T08:02:50.819 INFO:teuthology.orchestra.run.smithi067.stdout: available. 2024-02-10T08:02:50.819 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-10T08:02:50.825 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.tdJc6hFbSZ 2024-02-10T08:02:50.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub/grub.cfg /tmp/tmp.tdJc6hFbSZ 2024-02-10T08:02:50.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.tdJc6hFbSZ 2024-02-10T08:02:50.912 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.tdJc6hFbSZ is 9KB 2024-02-10T08:02:50.926 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.tdJc6hFbSZ 2024-02-10T08:02:50.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-02-10T08:02:50.990 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:02:50.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-02-10T08:02:50.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-02-10T08:02:51.036 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-94-generic 2024-02-10T08:02:51.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo update-grub 2024-02-10T08:02:52.056 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-02-10T08:02:52.083 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic-hwe-20.04 2024-02-10T08:02:52.148 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-02-10T08:02:52.356 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-02-10T08:02:52.356 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-02-10T08:02:52.513 INFO:teuthology.orchestra.run.smithi067.stderr:Sourcing file `/etc/default/grub' 2024-02-10T08:02:52.518 INFO:teuthology.orchestra.run.smithi067.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-02-10T08:02:52.530 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-02-10T08:02:52.582 INFO:teuthology.orchestra.run.smithi134.stdout:linux-image-generic-hwe-20.04 is already the newest version (5.15.0.94.104~20.04.50). 2024-02-10T08:02:52.582 INFO:teuthology.orchestra.run.smithi134.stdout:0 upgraded, 0 newly installed, 0 to remove and 287 not upgraded. 2024-02-10T08:02:52.584 DEBUG:teuthology.orchestra.run.smithi134:> dpkg -s linux-image-generic-hwe-20.04 2024-02-10T08:02:52.605 INFO:teuthology.orchestra.run.smithi134.stdout:Package: linux-image-generic-hwe-20.04 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Status: install ok installed 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Priority: optional 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Section: kernel 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Installed-Size: 20 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Maintainer: Ubuntu Kernel Team 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: amd64 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Source: linux-meta-hwe-5.15 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Version: 5.15.0.94.104~20.04.50 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-94), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Depends: linux-image-5.15.0-94-generic, linux-modules-extra-5.15.0-94-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Recommends: thermald 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout:Description: Generic Linux kernel image 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout: This package will always depend on the latest generic kernel image 2024-02-10T08:02:52.606 INFO:teuthology.orchestra.run.smithi134.stdout: available. 2024-02-10T08:02:52.607 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-02-10T08:02:52.611 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.8h2htDzP26 2024-02-10T08:02:52.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub/grub.cfg /tmp/tmp.8h2htDzP26 2024-02-10T08:02:52.621 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.8h2htDzP26 2024-02-10T08:02:52.709 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.8h2htDzP26 is 9KB 2024-02-10T08:02:52.723 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.8h2htDzP26 2024-02-10T08:02:52.729 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-02-10T08:02:52.766 INFO:teuthology.orchestra.run.smithi067.stderr:Found linux image: /boot/vmlinuz-5.15.0-94-generic 2024-02-10T08:02:52.778 INFO:teuthology.orchestra.run.smithi067.stderr:Found initrd image: /boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:52.784 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:02:52.785 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-02-10T08:02:52.785 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-02-10T08:02:52.813 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-94-generic 2024-02-10T08:02:52.813 DEBUG:teuthology.orchestra.run.smithi134:> sudo update-grub 2024-02-10T08:02:53.140 INFO:teuthology.orchestra.run.smithi067.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-02-10T08:02:53.151 INFO:teuthology.orchestra.run.smithi067.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-02-10T08:02:53.882 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-02-10T08:02:53.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-02-10T08:02:54.091 INFO:teuthology.orchestra.run.smithi134.stderr:Sourcing file `/etc/default/grub' 2024-02-10T08:02:54.101 INFO:teuthology.orchestra.run.smithi134.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-02-10T08:02:54.143 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-02-10T08:02:54.416 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.15.0-94-generic 2024-02-10T08:02:54.428 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.15.0-94-generic 2024-02-10T08:02:54.833 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-02-10T08:02:54.844 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-02-10T08:02:55.558 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-02-10T08:02:55.559 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-02-10T08:03:23.912 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-10T08:03:23.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-10T08:03:23.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:03:25.564 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-10T08:03:25.564 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-02-10T08:03:25.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:03:58.396 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-02-10T08:04:00.188 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-02-10T08:04:07.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-10T08:04:07.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:04:09.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-02-10T08:04:09.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:04:27.584 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.134 2024-02-10T08:04:39.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-02-10T08:04:39.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:04:39.849 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-02-10T08:04:40.431 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-02-10T08:04:40.431 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-94-generic"... 2024-02-10T08:04:40.431 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-02-10T08:04:40.480 INFO:teuthology.orchestra.run.smithi134.stdout:5.15.0-94-generic 2024-02-10T08:04:40.480 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-94-generic vs 5.15.0-94-generic 2024-02-10T08:04:40.480 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T08:04:40.481 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-02-10T08:04:41.481 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-10T08:04:41.481 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T08:04:41.854 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-02-10T08:04:41.877 DEBUG:teuthology.parallel:result is None 2024-02-10T08:05:07.407 DEBUG:teuthology.orchestra.remote:timed out 2024-02-10T08:05:19.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-10T08:05:19.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:05:19.537 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-02-10T08:05:20.062 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-10T08:05:20.062 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-94-generic"... 2024-02-10T08:05:20.063 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-02-10T08:05:20.116 INFO:teuthology.orchestra.run.smithi067.stdout:5.15.0-94-generic 2024-02-10T08:05:20.116 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-94-generic vs 5.15.0-94-generic 2024-02-10T08:05:20.116 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T08:05:20.116 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-02-10T08:05:21.117 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-10T08:05:21.117 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T08:05:21.192 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2024-02-10T08:05:21.220 DEBUG:teuthology.parallel:result is None 2024-02-10T08:05:21.221 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-10T08:05:21.231 INFO:teuthology.task.internal:Creating test directory... 2024-02-10T08:05:21.232 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T08:05:21.235 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T08:05:21.242 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-10T08:05:21.252 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-10T08:05:21.262 INFO:teuthology.task.internal:Creating archive directory... 2024-02-10T08:05:21.262 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T08:05:21.289 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T08:05:21.394 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-10T08:05:21.406 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-10T08:05:21.406 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T08:05:21.409 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T08:05:21.425 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T08:05:21.431 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T08:05:21.435 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T08:05:21.445 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T08:05:21.447 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-10T08:05:21.457 INFO:teuthology.task.internal:Configuring sudo... 2024-02-10T08:05:21.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T08:05:21.476 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T08:05:21.506 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-10T08:05:21.516 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-10T08:05:21.516 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T08:05:21.535 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T08:05:21.556 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T08:05:21.589 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T08:05:21.637 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:05:21.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T08:05:21.699 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T08:05:21.707 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T08:05:21.757 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:05:21.757 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T08:05:21.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-02-10T08:05:21.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-02-10T08:05:21.918 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-10T08:05:21.926 INFO:teuthology.task.internal:Starting timer... 2024-02-10T08:05:21.926 INFO:teuthology.run_tasks:Running task pcp... 2024-02-10T08:05:21.939 INFO:teuthology.run_tasks:Running task selinux... 2024-02-10T08:05:21.948 DEBUG:teuthology.task.selinux:Excluding smithi067: OS 'ubuntu' does not support SELinux 2024-02-10T08:05:21.948 DEBUG:teuthology.task.selinux:Excluding smithi134: OS 'ubuntu' does not support SELinux 2024-02-10T08:05:21.949 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-10T08:05:21.949 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-02-10T08:05:21.949 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-10T08:05:21.949 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-10T08:05:21.960 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-10T08:05:21.965 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-10T08:05:22.116 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-10T08:05:22.136 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-10T08:05:22.138 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-10T08:10:50.208 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-02-10T08:10:50.209 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-10T08:10:50.210 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:10:50.296 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-02-10T08:10:50.416 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-10T08:10:50.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-02-10T08:10:50.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T08:10:50.514 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-02-10T08:10:50.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-02-10T08:10:50.634 INFO:teuthology.run_tasks:Running task clock... 2024-02-10T08:10:50.645 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-10T08:10:50.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T08:10:50.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T08:10:50.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T08:10:50.648 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T08:10:50.677 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-02-10T08:10:50.678 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Command line: ntpd -gq 2024-02-10T08:10:50.678 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: proto: precision = 0.098 usec (-23) 2024-02-10T08:10:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-10T08:10:50.679 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-10T08:10:50.679 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: restrict ::: KOD does nothing without LIMITED. 2024-02-10T08:10:50.679 INFO:teuthology.orchestra.run.smithi067.stderr:restrict ::: KOD does nothing without LIMITED. 2024-02-10T08:10:50.680 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listen and drop on 0 v6wildcard [::]:123 2024-02-10T08:10:50.680 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-02-10T08:10:50.681 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listen normally on 2 lo 127.0.0.1:123 2024-02-10T08:10:50.681 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listen normally on 3 enp3s0f1 172.21.15.67:123 2024-02-10T08:10:50.681 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listen normally on 4 lo [::1]:123 2024-02-10T08:10:50.681 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:fc0b%5]:123 2024-02-10T08:10:50.681 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:50 ntpd[12756]: Listening on routing socket on fd #22 for interface updates 2024-02-10T08:10:50.707 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-02-10T08:10:50.707 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Command line: ntpd -gq 2024-02-10T08:10:50.708 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: proto: precision = 0.109 usec (-23) 2024-02-10T08:10:50.708 INFO:teuthology.orchestra.run.smithi134.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-10T08:10:50.709 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-10T08:10:50.709 INFO:teuthology.orchestra.run.smithi134.stderr:restrict ::: KOD does nothing without LIMITED. 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: restrict ::: KOD does nothing without LIMITED. 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listen and drop on 0 v6wildcard [::]:123 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listen normally on 2 lo 127.0.0.1:123 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listen normally on 3 enp3s0f1 172.21.15.134:123 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listen normally on 4 lo [::1]:123 2024-02-10T08:10:50.710 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6de9%5]:123 2024-02-10T08:10:50.711 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:50 ntpd[12769]: Listening on routing socket on fd #22 for interface updates 2024-02-10T08:10:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:51 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:51 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:51 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:51 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:51 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:51.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:51 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:51.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:51 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:52.678 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:52 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:52.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:52 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:52.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:52 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:52.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:52 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:52.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:52 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:52.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:52 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:53.678 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:53 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:53 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:53 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:53 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:53 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:53 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:53.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:53 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:54.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:54.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:54.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:54.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:54 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:54 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:54.708 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:54 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.678 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:55 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:55 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:55 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:55 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:55 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:55.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:55 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:55.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:55 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:56.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:56.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:56.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:56.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:56.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:56 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:57.679 INFO:teuthology.orchestra.run.smithi067.stdout:10 Feb 08:10:57 ntpd[12756]: ntpd: time slew -0.000009 s 2024-02-10T08:10:57.680 INFO:teuthology.orchestra.run.smithi067.stdout:ntpd: time slew -0.000009s 2024-02-10T08:10:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-02-10T08:10:57.680 INFO:teuthology.orchestra.run.smithi067.stderr:10 Feb 08:10:57 ntpd[12756]: can't open /var/log/ntpstats/loopstats.20240210: Permission denied 2024-02-10T08:10:57.708 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:57.708 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:57.708 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:57.708 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: can't open /var/log/ntpstats/rawstats.20240210: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: can't open /var/log/ntpstats/peerstats.20240210: Permission denied 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stdout:10 Feb 08:10:57 ntpd[12769]: ntpd: time slew +0.000057 s 2024-02-10T08:10:57.709 INFO:teuthology.orchestra.run.smithi134.stdout:ntpd: time slew +0.000057s 2024-02-10T08:10:57.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-02-10T08:10:57.710 INFO:teuthology.orchestra.run.smithi134.stderr:10 Feb 08:10:57 ntpd[12769]: can't open /var/log/ntpstats/loopstats.20240210: Permission denied 2024-02-10T08:10:57.734 INFO:teuthology.orchestra.run.smithi067.stdout: remote refid st t when poll reach delay offset jitter 2024-02-10T08:10:57.734 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================== 2024-02-10T08:10:57.734 INFO:teuthology.orchestra.run.smithi067.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.734 INFO:teuthology.orchestra.run.smithi067.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.735 INFO:teuthology.orchestra.run.smithi067.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.735 INFO:teuthology.orchestra.run.smithi067.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.776 INFO:teuthology.orchestra.run.smithi134.stdout: remote refid st t when poll reach delay offset jitter 2024-02-10T08:10:57.777 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================== 2024-02-10T08:10:57.777 INFO:teuthology.orchestra.run.smithi134.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.777 INFO:teuthology.orchestra.run.smithi134.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.777 INFO:teuthology.orchestra.run.smithi134.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.777 INFO:teuthology.orchestra.run.smithi134.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-10T08:10:57.778 INFO:teuthology.run_tasks:Running task install... 2024-02-10T08:10:57.788 DEBUG:teuthology.task.install:project ceph 2024-02-10T08:10:57.788 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'}, 'extra_packages': ['cephadm']} 2024-02-10T08:10:57.788 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1'} 2024-02-10T08:10:57.788 INFO:teuthology.task.install:Using flavor: default 2024-02-10T08:10:57.794 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-10T08:10:57.794 INFO:teuthology.task.install:extra packages: [] 2024-02-10T08:10:57.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo apt-key list | grep Ceph 2024-02-10T08:10:57.795 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-key list | grep Ceph 2024-02-10T08:10:57.898 INFO:teuthology.orchestra.run.smithi067.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-02-10T08:10:57.914 INFO:teuthology.orchestra.run.smithi134.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-02-10T08:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-02-10T08:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout:uid [ unknown] Ceph.com (release key) 2024-02-10T08:10:57.955 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-02-10T08:10:57.956 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:10:57.973 INFO:teuthology.orchestra.run.smithi134.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-02-10T08:10:57.973 INFO:teuthology.orchestra.run.smithi134.stdout:uid [ unknown] Ceph.com (release key) 2024-02-10T08:10:57.974 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-02-10T08:10:57.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:10:58.135 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default/ 2024-02-10T08:10:58.135 INFO:teuthology.task.install.deb:Package version is 16.2.14-521-g0e714d9a-1focal 2024-02-10T08:10:58.148 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default/ 2024-02-10T08:10:58.148 INFO:teuthology.task.install.deb:Package version is 16.2.14-521-g0e714d9a-1focal 2024-02-10T08:10:58.276 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:10:58.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-02-10T08:10:58.284 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:10:58.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-02-10T08:10:58.291 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-02-10T08:10:58.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo apt-get update 2024-02-10T08:10:58.503 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-02-10T08:10:58.543 INFO:teuthology.orchestra.run.smithi134.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal InRelease 2024-02-10T08:10:58.559 INFO:teuthology.orchestra.run.smithi067.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal InRelease 2024-02-10T08:10:58.567 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal Release [7,680 B] 2024-02-10T08:10:58.581 INFO:teuthology.orchestra.run.smithi067.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal Release [7,680 B] 2024-02-10T08:10:58.592 INFO:teuthology.orchestra.run.smithi134.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal Release.gpg 2024-02-10T08:10:58.594 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-02-10T08:10:58.616 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main i386 Packages [3,096 B] 2024-02-10T08:10:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease 2024-02-10T08:10:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-02-10T08:10:58.618 INFO:teuthology.orchestra.run.smithi067.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal Release.gpg 2024-02-10T08:10:58.642 INFO:teuthology.orchestra.run.smithi067.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2024-02-10T08:10:58.653 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 Packages [17.2 kB] 2024-02-10T08:10:58.676 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-02-10T08:10:58.677 INFO:teuthology.orchestra.run.smithi067.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main i386 Packages [3,096 B] 2024-02-10T08:10:58.687 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-02-10T08:10:58.764 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-02-10T08:10:58.769 INFO:teuthology.orchestra.run.smithi067.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-02-10T08:10:59.131 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 28.0 kB in 1s (44.7 kB/s) 2024-02-10T08:10:59.273 INFO:teuthology.orchestra.run.smithi067.stdout:Fetched 28.0 kB in 1s (36.8 kB/s) 2024-02-10T08:11:00.228 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-02-10T08:11:00.255 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-521-g0e714d9a-1focal cephadm=16.2.14-521-g0e714d9a-1focal ceph-mds=16.2.14-521-g0e714d9a-1focal ceph-mgr=16.2.14-521-g0e714d9a-1focal ceph-common=16.2.14-521-g0e714d9a-1focal ceph-fuse=16.2.14-521-g0e714d9a-1focal ceph-test=16.2.14-521-g0e714d9a-1focal radosgw=16.2.14-521-g0e714d9a-1focal python3-rados=16.2.14-521-g0e714d9a-1focal python3-rgw=16.2.14-521-g0e714d9a-1focal python3-cephfs=16.2.14-521-g0e714d9a-1focal python3-rbd=16.2.14-521-g0e714d9a-1focal libcephfs2=16.2.14-521-g0e714d9a-1focal libcephfs-dev=16.2.14-521-g0e714d9a-1focal librados2=16.2.14-521-g0e714d9a-1focal librbd1=16.2.14-521-g0e714d9a-1focal rbd-fuse=16.2.14-521-g0e714d9a-1focal 2024-02-10T08:11:00.326 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-02-10T08:11:00.412 INFO:teuthology.orchestra.run.smithi067.stdout:Reading package lists... 2024-02-10T08:11:00.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-521-g0e714d9a-1focal cephadm=16.2.14-521-g0e714d9a-1focal ceph-mds=16.2.14-521-g0e714d9a-1focal ceph-mgr=16.2.14-521-g0e714d9a-1focal ceph-common=16.2.14-521-g0e714d9a-1focal ceph-fuse=16.2.14-521-g0e714d9a-1focal ceph-test=16.2.14-521-g0e714d9a-1focal radosgw=16.2.14-521-g0e714d9a-1focal python3-rados=16.2.14-521-g0e714d9a-1focal python3-rgw=16.2.14-521-g0e714d9a-1focal python3-cephfs=16.2.14-521-g0e714d9a-1focal python3-rbd=16.2.14-521-g0e714d9a-1focal libcephfs2=16.2.14-521-g0e714d9a-1focal libcephfs-dev=16.2.14-521-g0e714d9a-1focal librados2=16.2.14-521-g0e714d9a-1focal librbd1=16.2.14-521-g0e714d9a-1focal rbd-fuse=16.2.14-521-g0e714d9a-1focal 2024-02-10T08:11:00.511 INFO:teuthology.orchestra.run.smithi067.stdout:Reading package lists... 2024-02-10T08:11:00.532 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-02-10T08:11:00.532 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-02-10T08:11:00.707 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages were automatically installed and are no longer required: 2024-02-10T08:11:00.708 INFO:teuthology.orchestra.run.smithi134.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-02-10T08:11:00.708 INFO:teuthology.orchestra.run.smithi134.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-10T08:11:00.708 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-02-10T08:11:00.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-02-10T08:11:00.709 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2024-02-10T08:11:00.709 INFO:teuthology.orchestra.run.smithi134.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2024-02-10T08:11:00.709 INFO:teuthology.orchestra.run.smithi134.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2024-02-10T08:11:00.709 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-02-10T08:11:00.710 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket python3-webtest python3-werkzeug socat xmlstarlet 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-02-10T08:11:00.711 INFO:teuthology.orchestra.run.smithi134.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-02-10T08:11:00.712 INFO:teuthology.orchestra.run.smithi134.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-02-10T08:11:00.712 INFO:teuthology.orchestra.run.smithi134.stdout:Recommended packages: 2024-02-10T08:11:00.713 INFO:teuthology.orchestra.run.smithi134.stdout: btrfs-tools 2024-02-10T08:11:00.722 INFO:teuthology.orchestra.run.smithi067.stdout:Building dependency tree... 2024-02-10T08:11:00.723 INFO:teuthology.orchestra.run.smithi067.stdout:Reading state information... 2024-02-10T08:11:00.830 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-02-10T08:11:00.830 INFO:teuthology.orchestra.run.smithi134.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-02-10T08:11:00.830 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-02-10T08:11:00.830 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2024-02-10T08:11:00.830 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 libjaeger libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2024-02-10T08:11:00.831 INFO:teuthology.orchestra.run.smithi134.stdout: libleveldb1d liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 2024-02-10T08:11:00.831 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli 2024-02-10T08:11:00.831 INFO:teuthology.orchestra.run.smithi134.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-02-10T08:11:00.831 INFO:teuthology.orchestra.run.smithi134.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-02-10T08:11:00.832 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-02-10T08:11:00.833 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2024-02-10T08:11:00.833 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages will be upgraded: 2024-02-10T08:11:00.833 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 librbd1 2024-02-10T08:11:00.917 INFO:teuthology.orchestra.run.smithi067.stdout:The following packages were automatically installed and are no longer required: 2024-02-10T08:11:00.917 INFO:teuthology.orchestra.run.smithi067.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-02-10T08:11:00.918 INFO:teuthology.orchestra.run.smithi067.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-10T08:11:00.918 INFO:teuthology.orchestra.run.smithi067.stdout:The following additional packages will be installed: 2024-02-10T08:11:00.918 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 libradosstriper1 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli python-pastedeploy-tpl 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-02-10T08:11:00.919 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket python3-webtest python3-werkzeug socat xmlstarlet 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout:Suggested packages: 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-02-10T08:11:00.920 INFO:teuthology.orchestra.run.smithi067.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout:Recommended packages: 2024-02-10T08:11:00.921 INFO:teuthology.orchestra.run.smithi067.stdout: btrfs-tools 2024-02-10T08:11:00.972 INFO:teuthology.orchestra.run.smithi134.stdout:2 upgraded, 98 newly installed, 0 to remove and 266 not upgraded. 2024-02-10T08:11:00.972 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 160 MB of archives. 2024-02-10T08:11:00.972 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 701 MB of additional disk space will be used. 2024-02-10T08:11:00.972 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.14-521-g0e714d9a-1focal [3,960 B] 2024-02-10T08:11:01.025 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-02-10T08:11:01.027 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.14-521-g0e714d9a-1focal [3,394 kB] 2024-02-10T08:11:01.040 INFO:teuthology.orchestra.run.smithi067.stdout:The following NEW packages will be installed: 2024-02-10T08:11:01.040 INFO:teuthology.orchestra.run.smithi067.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-02-10T08:11:01.040 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-02-10T08:11:01.040 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2024-02-10T08:11:01.040 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 libjaeger libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2024-02-10T08:11:01.040 INFO:teuthology.orchestra.run.smithi067.stdout: libleveldb1d liblttng-ust0 liblua5.3-0 liboath0 libonig5 librabbitmq4 2024-02-10T08:11:01.041 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph nvme-cli 2024-02-10T08:11:01.041 INFO:teuthology.orchestra.run.smithi067.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-02-10T08:11:01.041 INFO:teuthology.orchestra.run.smithi067.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-02-10T08:11:01.041 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-02-10T08:11:01.041 INFO:teuthology.orchestra.run.smithi067.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-02-10T08:11:01.042 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2024-02-10T08:11:01.043 INFO:teuthology.orchestra.run.smithi067.stdout:The following packages will be upgraded: 2024-02-10T08:11:01.043 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 librbd1 2024-02-10T08:11:01.193 INFO:teuthology.orchestra.run.smithi067.stdout:2 upgraded, 98 newly installed, 0 to remove and 266 not upgraded. 2024-02-10T08:11:01.193 INFO:teuthology.orchestra.run.smithi067.stdout:Need to get 160 MB of archives. 2024-02-10T08:11:01.193 INFO:teuthology.orchestra.run.smithi067.stdout:After this operation, 701 MB of additional disk space will be used. 2024-02-10T08:11:01.194 INFO:teuthology.orchestra.run.smithi067.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libjaeger amd64 16.2.14-521-g0e714d9a-1focal [3,960 B] 2024-02-10T08:11:01.236 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.14-521-g0e714d9a-1focal [3,395 kB] 2024-02-10T08:11:01.237 INFO:teuthology.orchestra.run.smithi067.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-02-10T08:11:01.304 INFO:teuthology.orchestra.run.smithi067.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 16.2.14-521-g0e714d9a-1focal [3,394 kB] 2024-02-10T08:11:01.349 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.14-521-g0e714d9a-1focal [730 kB] 2024-02-10T08:11:01.374 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.14-521-g0e714d9a-1focal [414 kB] 2024-02-10T08:11:01.387 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.14-521-g0e714d9a-1focal [22.2 kB] 2024-02-10T08:11:01.388 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.14-521-g0e714d9a-1focal [212 kB] 2024-02-10T08:11:01.395 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.14-521-g0e714d9a-1focal [38.4 kB] 2024-02-10T08:11:01.396 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.14-521-g0e714d9a-1focal [385 kB] 2024-02-10T08:11:01.409 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.14-521-g0e714d9a-1focal [3,707 kB] 2024-02-10T08:11:01.431 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-02-10T08:11:01.438 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-02-10T08:11:01.482 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-02-10T08:11:01.513 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-02-10T08:11:01.544 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.14-521-g0e714d9a-1focal [116 kB] 2024-02-10T08:11:01.548 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.14-521-g0e714d9a-1focal [448 kB] 2024-02-10T08:11:01.555 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.14-521-g0e714d9a-1focal [21.8 MB] 2024-02-10T08:11:01.623 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2024-02-10T08:11:01.640 INFO:teuthology.orchestra.run.smithi067.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-02-10T08:11:01.646 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-02-10T08:11:01.648 INFO:teuthology.orchestra.run.smithi067.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-02-10T08:11:01.653 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.2 [211 kB] 2024-02-10T08:11:01.684 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-02-10T08:11:01.689 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-02-10T08:11:01.690 INFO:teuthology.orchestra.run.smithi067.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-02-10T08:11:01.692 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-02-10T08:11:01.722 INFO:teuthology.orchestra.run.smithi067.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-02-10T08:11:01.831 INFO:teuthology.orchestra.run.smithi067.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2024-02-10T08:11:01.854 INFO:teuthology.orchestra.run.smithi067.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-02-10T08:11:01.860 INFO:teuthology.orchestra.run.smithi067.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.2 [211 kB] 2024-02-10T08:11:01.870 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-02-10T08:11:01.891 INFO:teuthology.orchestra.run.smithi067.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-02-10T08:11:01.896 INFO:teuthology.orchestra.run.smithi067.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-02-10T08:11:01.899 INFO:teuthology.orchestra.run.smithi067.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-02-10T08:11:02.048 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-02-10T08:11:02.077 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-02-10T08:11:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-02-10T08:11:02.107 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-02-10T08:11:02.224 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-02-10T08:11:02.234 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-02-10T08:11:02.238 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-02-10T08:11:02.245 INFO:teuthology.orchestra.run.smithi067.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-02-10T08:11:02.275 INFO:teuthology.orchestra.run.smithi067.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-02-10T08:11:02.299 INFO:teuthology.orchestra.run.smithi067.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-02-10T08:11:02.306 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-02-10T08:11:02.372 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-02-10T08:11:02.405 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-02-10T08:11:02.405 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.14-521-g0e714d9a-1focal [4,731 kB] 2024-02-10T08:11:02.406 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-02-10T08:11:02.415 INFO:teuthology.orchestra.run.smithi067.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-02-10T08:11:02.428 INFO:teuthology.orchestra.run.smithi067.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-02-10T08:11:02.432 INFO:teuthology.orchestra.run.smithi067.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-02-10T08:11:02.471 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.14-521-g0e714d9a-1focal [204 kB] 2024-02-10T08:11:02.476 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.14-521-g0e714d9a-1focal [126 kB] 2024-02-10T08:11:02.480 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.14-521-g0e714d9a-1focal [1,551 kB] 2024-02-10T08:11:02.499 INFO:teuthology.orchestra.run.smithi067.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-02-10T08:11:02.515 INFO:teuthology.orchestra.run.smithi067.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-02-10T08:11:02.534 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.14-521-g0e714d9a-1focal [4,015 kB] 2024-02-10T08:11:02.550 INFO:teuthology.orchestra.run.smithi067.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-02-10T08:11:02.571 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-02-10T08:11:02.577 INFO:teuthology.orchestra.run.smithi067.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-02-10T08:11:02.672 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.14-521-g0e714d9a-1focal [21.2 MB] 2024-02-10T08:11:02.745 INFO:teuthology.orchestra.run.smithi067.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-02-10T08:11:03.103 INFO:teuthology.orchestra.run.smithi134.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-02-10T08:11:03.104 INFO:teuthology.orchestra.run.smithi134.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-02-10T08:11:03.107 INFO:teuthology.orchestra.run.smithi134.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-02-10T08:11:03.110 INFO:teuthology.orchestra.run.smithi134.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-02-10T08:11:03.121 INFO:teuthology.orchestra.run.smithi134.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2024-02-10T08:11:03.145 INFO:teuthology.orchestra.run.smithi134.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.2 [95.9 kB] 2024-02-10T08:11:03.157 INFO:teuthology.orchestra.run.smithi134.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-02-10T08:11:03.158 INFO:teuthology.orchestra.run.smithi134.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-02-10T08:11:03.193 INFO:teuthology.orchestra.run.smithi134.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-02-10T08:11:03.205 INFO:teuthology.orchestra.run.smithi134.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-02-10T08:11:03.269 INFO:teuthology.orchestra.run.smithi067.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-02-10T08:11:03.270 INFO:teuthology.orchestra.run.smithi067.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-02-10T08:11:03.272 INFO:teuthology.orchestra.run.smithi067.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-02-10T08:11:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-02-10T08:11:03.287 INFO:teuthology.orchestra.run.smithi067.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2024-02-10T08:11:03.311 INFO:teuthology.orchestra.run.smithi067.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.2 [95.9 kB] 2024-02-10T08:11:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-02-10T08:11:03.325 INFO:teuthology.orchestra.run.smithi067.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-02-10T08:11:03.359 INFO:teuthology.orchestra.run.smithi067.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-02-10T08:11:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-02-10T08:11:03.380 INFO:teuthology.orchestra.run.smithi134.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-02-10T08:11:03.455 INFO:teuthology.orchestra.run.smithi134.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.14-521-g0e714d9a-1focal [4,052 B] 2024-02-10T08:11:03.463 INFO:teuthology.orchestra.run.smithi134.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-02-10T08:11:03.523 INFO:teuthology.orchestra.run.smithi134.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.14-521-g0e714d9a-1focal [836 kB] 2024-02-10T08:11:03.529 INFO:teuthology.orchestra.run.smithi134.stdout:Get:57 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.14-521-g0e714d9a-1focal [2,363 kB] 2024-02-10T08:11:03.546 INFO:teuthology.orchestra.run.smithi134.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.14-521-g0e714d9a-1focal [77.4 kB] 2024-02-10T08:11:03.546 INFO:teuthology.orchestra.run.smithi134.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.14-521-g0e714d9a-1focal [100 kB] 2024-02-10T08:11:03.547 INFO:teuthology.orchestra.run.smithi067.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-02-10T08:11:03.547 INFO:teuthology.orchestra.run.smithi134.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.14-521-g0e714d9a-1focal [2,824 kB] 2024-02-10T08:11:03.554 INFO:teuthology.orchestra.run.smithi134.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-02-10T08:11:03.598 INFO:teuthology.orchestra.run.smithi067.stdout:Get:37 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 16.2.14-521-g0e714d9a-1focal [3,395 kB] 2024-02-10T08:11:03.624 INFO:teuthology.orchestra.run.smithi134.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.14-521-g0e714d9a-1focal [7,874 kB] 2024-02-10T08:11:03.630 INFO:teuthology.orchestra.run.smithi067.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-02-10T08:11:03.721 INFO:teuthology.orchestra.run.smithi067.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-02-10T08:11:03.896 INFO:teuthology.orchestra.run.smithi134.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.14-521-g0e714d9a-1focal [4,168 B] 2024-02-10T08:11:03.897 INFO:teuthology.orchestra.run.smithi134.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.14-521-g0e714d9a-1focal [45.8 MB] 2024-02-10T08:11:04.294 INFO:teuthology.orchestra.run.smithi134.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-02-10T08:11:04.367 INFO:teuthology.orchestra.run.smithi134.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-02-10T08:11:04.369 INFO:teuthology.orchestra.run.smithi134.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-02-10T08:11:04.410 INFO:teuthology.orchestra.run.smithi134.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-02-10T08:11:04.410 INFO:teuthology.orchestra.run.smithi134.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-02-10T08:11:04.410 INFO:teuthology.orchestra.run.smithi134.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-02-10T08:11:04.411 INFO:teuthology.orchestra.run.smithi134.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-02-10T08:11:04.411 INFO:teuthology.orchestra.run.smithi134.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-02-10T08:11:04.483 INFO:teuthology.orchestra.run.smithi067.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-02-10T08:11:04.536 INFO:teuthology.orchestra.run.smithi067.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-02-10T08:11:04.565 INFO:teuthology.orchestra.run.smithi067.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-02-10T08:11:04.576 INFO:teuthology.orchestra.run.smithi134.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-02-10T08:11:04.577 INFO:teuthology.orchestra.run.smithi067.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-02-10T08:11:04.577 INFO:teuthology.orchestra.run.smithi067.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-02-10T08:11:04.578 INFO:teuthology.orchestra.run.smithi067.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-02-10T08:11:04.578 INFO:teuthology.orchestra.run.smithi067.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-02-10T08:11:04.578 INFO:teuthology.orchestra.run.smithi067.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-02-10T08:11:04.660 INFO:teuthology.orchestra.run.smithi134.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-02-10T08:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-02-10T08:11:04.827 INFO:teuthology.orchestra.run.smithi067.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-02-10T08:11:04.840 INFO:teuthology.orchestra.run.smithi134.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-02-10T08:11:04.855 INFO:teuthology.orchestra.run.smithi134.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-02-10T08:11:04.888 INFO:teuthology.orchestra.run.smithi134.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-02-10T08:11:05.007 INFO:teuthology.orchestra.run.smithi067.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-02-10T08:11:05.018 INFO:teuthology.orchestra.run.smithi134.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-02-10T08:11:05.022 INFO:teuthology.orchestra.run.smithi067.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-02-10T08:11:05.054 INFO:teuthology.orchestra.run.smithi134.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-02-10T08:11:05.055 INFO:teuthology.orchestra.run.smithi067.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-02-10T08:11:05.078 INFO:teuthology.orchestra.run.smithi134.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-02-10T08:11:05.086 INFO:teuthology.orchestra.run.smithi134.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-02-10T08:11:05.129 INFO:teuthology.orchestra.run.smithi134.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-02-10T08:11:05.156 INFO:teuthology.orchestra.run.smithi134.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-02-10T08:11:05.185 INFO:teuthology.orchestra.run.smithi067.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-02-10T08:11:05.221 INFO:teuthology.orchestra.run.smithi067.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-02-10T08:11:05.245 INFO:teuthology.orchestra.run.smithi067.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-02-10T08:11:05.253 INFO:teuthology.orchestra.run.smithi067.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-02-10T08:11:05.296 INFO:teuthology.orchestra.run.smithi067.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-02-10T08:11:05.323 INFO:teuthology.orchestra.run.smithi067.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-02-10T08:11:05.342 INFO:teuthology.orchestra.run.smithi134.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-02-10T08:11:05.425 INFO:teuthology.orchestra.run.smithi134.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-02-10T08:11:05.509 INFO:teuthology.orchestra.run.smithi067.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-02-10T08:11:05.592 INFO:teuthology.orchestra.run.smithi067.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-02-10T08:11:05.599 INFO:teuthology.orchestra.run.smithi134.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-02-10T08:11:05.599 INFO:teuthology.orchestra.run.smithi134.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-02-10T08:11:05.648 INFO:teuthology.orchestra.run.smithi067.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 16.2.14-521-g0e714d9a-1focal [730 kB] 2024-02-10T08:11:05.666 INFO:teuthology.orchestra.run.smithi134.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.14-521-g0e714d9a-1focal [21.5 kB] 2024-02-10T08:11:05.708 INFO:teuthology.orchestra.run.smithi067.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-02-10T08:11:05.731 INFO:teuthology.orchestra.run.smithi134.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-02-10T08:11:05.731 INFO:teuthology.orchestra.run.smithi134.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.14-521-g0e714d9a-1focal [11.3 MB] 2024-02-10T08:11:05.736 INFO:teuthology.orchestra.run.smithi067.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-02-10T08:11:05.860 INFO:teuthology.orchestra.run.smithi067.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-02-10T08:11:05.886 INFO:teuthology.orchestra.run.smithi067.stdout:Get:65 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 16.2.14-521-g0e714d9a-1focal [414 kB] 2024-02-10T08:11:05.992 INFO:teuthology.orchestra.run.smithi134.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-02-10T08:11:05.993 INFO:teuthology.orchestra.run.smithi134.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-02-10T08:11:05.997 INFO:teuthology.orchestra.run.smithi134.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-02-10T08:11:05.998 INFO:teuthology.orchestra.run.smithi134.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-02-10T08:11:05.999 INFO:teuthology.orchestra.run.smithi134.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.14-521-g0e714d9a-1focal [78.1 kB] 2024-02-10T08:11:06.008 INFO:teuthology.orchestra.run.smithi134.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-02-10T08:11:06.013 INFO:teuthology.orchestra.run.smithi134.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-02-10T08:11:06.046 INFO:teuthology.orchestra.run.smithi067.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 16.2.14-521-g0e714d9a-1focal [22.2 kB] 2024-02-10T08:11:06.046 INFO:teuthology.orchestra.run.smithi067.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 16.2.14-521-g0e714d9a-1focal [212 kB] 2024-02-10T08:11:06.091 INFO:teuthology.orchestra.run.smithi067.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 16.2.14-521-g0e714d9a-1focal [38.4 kB] 2024-02-10T08:11:06.093 INFO:teuthology.orchestra.run.smithi067.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 16.2.14-521-g0e714d9a-1focal [385 kB] 2024-02-10T08:11:06.149 INFO:teuthology.orchestra.run.smithi067.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-02-10T08:11:06.150 INFO:teuthology.orchestra.run.smithi067.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-02-10T08:11:06.155 INFO:teuthology.orchestra.run.smithi067.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-02-10T08:11:06.155 INFO:teuthology.orchestra.run.smithi067.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-02-10T08:11:06.162 INFO:teuthology.orchestra.run.smithi067.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 16.2.14-521-g0e714d9a-1focal [3,707 kB] 2024-02-10T08:11:06.165 INFO:teuthology.orchestra.run.smithi067.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-02-10T08:11:06.170 INFO:teuthology.orchestra.run.smithi067.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-02-10T08:11:06.179 INFO:teuthology.orchestra.run.smithi134.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-02-10T08:11:06.336 INFO:teuthology.orchestra.run.smithi067.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-02-10T08:11:06.346 INFO:teuthology.orchestra.run.smithi134.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-02-10T08:11:06.502 INFO:teuthology.orchestra.run.smithi067.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-02-10T08:11:06.598 INFO:teuthology.orchestra.run.smithi134.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-02-10T08:11:06.754 INFO:teuthology.orchestra.run.smithi067.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-02-10T08:11:06.773 INFO:teuthology.orchestra.run.smithi067.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 16.2.14-521-g0e714d9a-1focal [116 kB] 2024-02-10T08:11:06.798 INFO:teuthology.orchestra.run.smithi067.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 16.2.14-521-g0e714d9a-1focal [448 kB] 2024-02-10T08:11:06.870 INFO:teuthology.orchestra.run.smithi067.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 16.2.14-521-g0e714d9a-1focal [21.8 MB] 2024-02-10T08:11:07.290 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 160 MB in 6s (27.7 MB/s) 2024-02-10T08:11:07.380 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjaeger. 2024-02-10T08:11:07.425 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112645 files and directories currently installed.) 2024-02-10T08:11:07.438 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../00-libjaeger_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:07.479 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjaeger (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:07.831 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-02-10T08:11:07.844 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../01-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-02-10T08:11:07.922 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-02-10T08:11:08.257 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../02-librbd1_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:08.384 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librbd1 (16.2.14-521-g0e714d9a-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-02-10T08:11:09.070 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../03-librados2_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:09.198 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librados2 (16.2.14-521-g0e714d9a-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-02-10T08:11:09.656 INFO:teuthology.orchestra.run.smithi067.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 16.2.14-521-g0e714d9a-1focal [4,731 kB] 2024-02-10T08:11:09.860 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libcephfs2. 2024-02-10T08:11:09.876 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../04-libcephfs2_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:09.918 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libcephfs2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:10.085 INFO:teuthology.orchestra.run.smithi067.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 16.2.14-521-g0e714d9a-1focal [204 kB] 2024-02-10T08:11:10.106 INFO:teuthology.orchestra.run.smithi067.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 16.2.14-521-g0e714d9a-1focal [126 kB] 2024-02-10T08:11:10.110 INFO:teuthology.orchestra.run.smithi067.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 16.2.14-521-g0e714d9a-1focal [1,551 kB] 2024-02-10T08:11:10.254 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rados. 2024-02-10T08:11:10.266 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../05-python3-rados_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:10.271 INFO:teuthology.orchestra.run.smithi067.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 16.2.14-521-g0e714d9a-1focal [4,015 kB] 2024-02-10T08:11:10.304 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rados (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:10.632 INFO:teuthology.orchestra.run.smithi067.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 16.2.14-521-g0e714d9a-1focal [21.2 MB] 2024-02-10T08:11:10.648 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-02-10T08:11:10.663 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../06-python3-ceph-argparse_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:10.731 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-ceph-argparse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:11.017 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cephfs. 2024-02-10T08:11:11.030 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../07-python3-cephfs_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:11.067 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cephfs (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:11.403 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-ceph-common. 2024-02-10T08:11:11.416 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../08-python3-ceph-common_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:11.461 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:11.773 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-prettytable. 2024-02-10T08:11:11.786 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../09-python3-prettytable_0.7.2-5_all.deb ... 2024-02-10T08:11:11.822 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-02-10T08:11:12.100 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rbd. 2024-02-10T08:11:12.113 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../10-python3-rbd_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:12.158 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rbd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:12.499 INFO:teuthology.orchestra.run.smithi067.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 16.2.14-521-g0e714d9a-1focal [4,052 B] 2024-02-10T08:11:12.499 INFO:teuthology.orchestra.run.smithi067.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 16.2.14-521-g0e714d9a-1focal [836 kB] 2024-02-10T08:11:12.526 INFO:teuthology.orchestra.run.smithi067.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 16.2.14-521-g0e714d9a-1focal [2,363 kB] 2024-02-10T08:11:12.561 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-02-10T08:11:12.575 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../11-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-02-10T08:11:12.627 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-02-10T08:11:12.700 INFO:teuthology.orchestra.run.smithi067.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 16.2.14-521-g0e714d9a-1focal [77.4 kB] 2024-02-10T08:11:12.707 INFO:teuthology.orchestra.run.smithi067.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 16.2.14-521-g0e714d9a-1focal [100 kB] 2024-02-10T08:11:12.717 INFO:teuthology.orchestra.run.smithi067.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 16.2.14-521-g0e714d9a-1focal [2,824 kB] 2024-02-10T08:11:12.931 INFO:teuthology.orchestra.run.smithi067.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 16.2.14-521-g0e714d9a-1focal [7,874 kB] 2024-02-10T08:11:12.981 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-02-10T08:11:12.994 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../12-librabbitmq4_0.10.0-1_amd64.deb ... 2024-02-10T08:11:13.039 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-02-10T08:11:13.375 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-02-10T08:11:13.391 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../13-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-02-10T08:11:13.425 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-02-10T08:11:13.497 INFO:teuthology.orchestra.run.smithi067.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 16.2.14-521-g0e714d9a-1focal [4,168 B] 2024-02-10T08:11:13.497 INFO:teuthology.orchestra.run.smithi067.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 16.2.14-521-g0e714d9a-1focal [45.8 MB] 2024-02-10T08:11:13.828 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librgw2. 2024-02-10T08:11:13.841 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../14-librgw2_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:13.894 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librgw2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:14.497 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rgw. 2024-02-10T08:11:14.511 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../15-python3-rgw_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:14.580 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:14.908 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libleveldb1d:amd64. 2024-02-10T08:11:14.921 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../16-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2024-02-10T08:11:14.958 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2024-02-10T08:11:15.310 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liboath0. 2024-02-10T08:11:15.324 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../17-liboath0_2.6.1-1.3_amd64.deb ... 2024-02-10T08:11:15.368 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-02-10T08:11:15.688 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libradosstriper1. 2024-02-10T08:11:15.703 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../18-libradosstriper1_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:15.746 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libradosstriper1 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:16.007 INFO:teuthology.orchestra.run.smithi067.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 16.2.14-521-g0e714d9a-1focal [21.5 kB] 2024-02-10T08:11:16.082 INFO:teuthology.orchestra.run.smithi067.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 16.2.14-521-g0e714d9a-1focal [11.3 MB] 2024-02-10T08:11:16.200 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-common. 2024-02-10T08:11:16.215 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../19-ceph-common_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:16.258 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:16.410 INFO:teuthology.orchestra.run.smithi067.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri10-testing-2024-02-08-0854-pacific/0e714d9a4bd2a821113e6318adb87bd06cf81ec1/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 16.2.14-521-g0e714d9a-1focal [78.1 kB] 2024-02-10T08:11:17.081 INFO:teuthology.orchestra.run.smithi067.stdout:Fetched 160 MB in 15s (10.4 MB/s) 2024-02-10T08:11:17.175 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libjaeger. 2024-02-10T08:11:17.223 INFO:teuthology.orchestra.run.smithi067.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112645 files and directories currently installed.) 2024-02-10T08:11:17.226 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../00-libjaeger_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:17.265 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libjaeger (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:17.585 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-02-10T08:11:17.599 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../01-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-02-10T08:11:17.685 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-02-10T08:11:18.003 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../02-librbd1_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:18.105 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking librbd1 (16.2.14-521-g0e714d9a-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-02-10T08:11:18.330 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-base. 2024-02-10T08:11:18.343 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../20-ceph-base_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:18.413 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-base (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:18.756 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../03-librados2_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:18.976 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking librados2 (16.2.14-521-g0e714d9a-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-02-10T08:11:19.409 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cryptography. 2024-02-10T08:11:19.417 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../21-python3-cryptography_2.8-3ubuntu0.2_amd64.deb ... 2024-02-10T08:11:19.459 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.2) ... 2024-02-10T08:11:19.614 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libcephfs2. 2024-02-10T08:11:19.628 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../04-libcephfs2_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:19.672 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libcephfs2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:19.820 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-openssl. 2024-02-10T08:11:19.835 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../22-python3-openssl_19.0.0-1build1_all.deb ... 2024-02-10T08:11:19.878 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-02-10T08:11:20.000 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-rados. 2024-02-10T08:11:20.013 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../05-python3-rados_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:20.050 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-rados (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:20.123 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-02-10T08:11:20.138 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../23-ceph-mgr-modules-core_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:20.181 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:20.403 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-02-10T08:11:20.419 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../06-python3-ceph-argparse_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:20.461 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-ceph-argparse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:20.552 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-bcrypt. 2024-02-10T08:11:20.567 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../24-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-02-10T08:11:20.610 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-02-10T08:11:20.764 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-cephfs. 2024-02-10T08:11:20.779 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../07-python3-cephfs_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:20.814 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-cephfs (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:20.896 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cherrypy3. 2024-02-10T08:11:20.909 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../25-python3-cherrypy3_8.9.1-6_all.deb ... 2024-02-10T08:11:20.946 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-02-10T08:11:21.108 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-ceph-common. 2024-02-10T08:11:21.123 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../08-python3-ceph-common_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:21.166 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:21.291 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jwt. 2024-02-10T08:11:21.307 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../26-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-02-10T08:11:21.350 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-02-10T08:11:21.478 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-prettytable. 2024-02-10T08:11:21.492 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../09-python3-prettytable_0.7.2-5_all.deb ... 2024-02-10T08:11:21.536 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-02-10T08:11:21.661 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-logutils. 2024-02-10T08:11:21.674 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../27-python3-logutils_0.3.3-7_all.deb ... 2024-02-10T08:11:21.719 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-02-10T08:11:21.789 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-rbd. 2024-02-10T08:11:21.803 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../10-python3-rbd_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:21.847 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-rbd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:22.013 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-markupsafe. 2024-02-10T08:11:22.026 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../28-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-02-10T08:11:22.063 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-02-10T08:11:22.208 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-02-10T08:11:22.223 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../11-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-02-10T08:11:22.266 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-02-10T08:11:22.366 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-mako. 2024-02-10T08:11:22.380 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../29-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-02-10T08:11:22.424 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-02-10T08:11:22.577 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-02-10T08:11:22.591 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../12-librabbitmq4_0.10.0-1_amd64.deb ... 2024-02-10T08:11:22.636 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-02-10T08:11:22.744 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-simplegeneric. 2024-02-10T08:11:22.758 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../30-python3-simplegeneric_0.8.1-3_all.deb ... 2024-02-10T08:11:22.802 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-02-10T08:11:22.981 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-02-10T08:11:22.995 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../13-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-02-10T08:11:23.038 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-02-10T08:11:23.155 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-singledispatch. 2024-02-10T08:11:23.168 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../31-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-02-10T08:11:23.221 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-02-10T08:11:23.383 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package librgw2. 2024-02-10T08:11:23.399 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../14-librgw2_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:23.441 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking librgw2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:23.524 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webob. 2024-02-10T08:11:23.538 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../32-python3-webob_1%3a1.8.5-2_all.deb ... 2024-02-10T08:11:23.582 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-02-10T08:11:23.969 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-soupsieve. 2024-02-10T08:11:23.982 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../33-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-02-10T08:11:24.043 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-02-10T08:11:24.044 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-rgw. 2024-02-10T08:11:24.059 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../15-python3-rgw_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:24.102 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-rgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:24.388 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-bs4. 2024-02-10T08:11:24.401 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../34-python3-bs4_4.8.2-1_all.deb ... 2024-02-10T08:11:24.422 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libleveldb1d:amd64. 2024-02-10T08:11:24.435 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../16-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2024-02-10T08:11:24.446 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-02-10T08:11:24.463 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2024-02-10T08:11:24.775 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package liboath0. 2024-02-10T08:11:24.788 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../17-liboath0_2.6.1-1.3_amd64.deb ... 2024-02-10T08:11:24.816 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-waitress. 2024-02-10T08:11:24.824 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../35-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-02-10T08:11:24.825 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-02-10T08:11:24.868 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-02-10T08:11:25.136 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libradosstriper1. 2024-02-10T08:11:25.149 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../18-libradosstriper1_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:25.177 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-tempita. 2024-02-10T08:11:25.185 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../36-python3-tempita_0.5.2-6_all.deb ... 2024-02-10T08:11:25.193 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libradosstriper1 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:25.227 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-02-10T08:11:25.512 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-paste. 2024-02-10T08:11:25.521 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../37-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-02-10T08:11:25.562 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-02-10T08:11:25.631 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-common. 2024-02-10T08:11:25.644 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../19-ceph-common_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:25.688 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:25.866 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-02-10T08:11:25.880 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../38-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-02-10T08:11:25.924 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-02-10T08:11:26.251 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pastedeploy. 2024-02-10T08:11:26.267 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../39-python3-pastedeploy_2.1.0-1_all.deb ... 2024-02-10T08:11:26.310 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-02-10T08:11:26.638 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webtest. 2024-02-10T08:11:26.651 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../40-python3-webtest_2.0.32-2_all.deb ... 2024-02-10T08:11:26.695 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-02-10T08:11:27.040 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pecan. 2024-02-10T08:11:27.049 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../41-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-02-10T08:11:27.090 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-02-10T08:11:27.428 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-werkzeug. 2024-02-10T08:11:27.443 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../42-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2024-02-10T08:11:27.486 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-02-10T08:11:27.711 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-base. 2024-02-10T08:11:27.724 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../20-ceph-base_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:27.781 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-base (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:27.868 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-02-10T08:11:27.880 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../43-libsqlite3-mod-ceph_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:27.923 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:28.242 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr. 2024-02-10T08:11:28.256 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../44-ceph-mgr_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:28.301 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:28.648 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-cryptography. 2024-02-10T08:11:28.663 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../21-python3-cryptography_2.8-3ubuntu0.2_amd64.deb ... 2024-02-10T08:11:28.706 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.2) ... 2024-02-10T08:11:28.770 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mon. 2024-02-10T08:11:28.784 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../45-ceph-mon_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:28.828 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mon (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:29.085 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-openssl. 2024-02-10T08:11:29.098 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../22-python3-openssl_19.0.0-1build1_all.deb ... 2024-02-10T08:11:29.143 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-02-10T08:11:29.437 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-02-10T08:11:29.452 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../23-ceph-mgr-modules-core_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:29.495 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:29.540 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-osd. 2024-02-10T08:11:29.553 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../46-ceph-osd_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:29.590 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-osd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:29.926 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-bcrypt. 2024-02-10T08:11:29.939 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../24-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-02-10T08:11:29.984 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-02-10T08:11:30.353 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-cherrypy3. 2024-02-10T08:11:30.366 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../25-python3-cherrypy3_8.9.1-6_all.deb ... 2024-02-10T08:11:30.411 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-02-10T08:11:30.757 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-jwt. 2024-02-10T08:11:30.771 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../26-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-02-10T08:11:30.814 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-02-10T08:11:31.126 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-logutils. 2024-02-10T08:11:31.139 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../27-python3-logutils_0.3.3-7_all.deb ... 2024-02-10T08:11:31.184 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-02-10T08:11:31.487 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-markupsafe. 2024-02-10T08:11:31.501 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../28-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-02-10T08:11:31.545 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-02-10T08:11:31.570 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph. 2024-02-10T08:11:31.584 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../47-ceph_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:31.628 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:31.848 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-mako. 2024-02-10T08:11:31.862 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../29-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-02-10T08:11:31.906 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-02-10T08:11:31.931 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-fuse. 2024-02-10T08:11:31.946 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../48-ceph-fuse_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:31.989 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:32.227 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-simplegeneric. 2024-02-10T08:11:32.235 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../30-python3-simplegeneric_0.8.1-3_all.deb ... 2024-02-10T08:11:32.276 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-02-10T08:11:32.401 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mds. 2024-02-10T08:11:32.414 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../49-ceph-mds_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:32.458 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mds (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:32.537 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-singledispatch. 2024-02-10T08:11:32.545 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../31-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-02-10T08:11:32.587 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-02-10T08:11:32.873 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-webob. 2024-02-10T08:11:32.886 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../32-python3-webob_1%3a1.8.5-2_all.deb ... 2024-02-10T08:11:32.948 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-02-10T08:11:32.995 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package cephadm. 2024-02-10T08:11:33.010 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../50-cephadm_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:33.053 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:33.276 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-soupsieve. 2024-02-10T08:11:33.289 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../33-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-02-10T08:11:33.334 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-02-10T08:11:33.364 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jinja2. 2024-02-10T08:11:33.377 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../51-python3-jinja2_2.10.1-2ubuntu0.2_all.deb ... 2024-02-10T08:11:33.431 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-02-10T08:11:33.620 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-bs4. 2024-02-10T08:11:33.635 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../34-python3-bs4_4.8.2-1_all.deb ... 2024-02-10T08:11:33.678 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-02-10T08:11:33.775 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-02-10T08:11:33.789 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../52-ceph-mgr-cephadm_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:33.833 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:34.034 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-waitress. 2024-02-10T08:11:34.047 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../35-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-02-10T08:11:34.095 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-02-10T08:11:34.145 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-repoze.lru. 2024-02-10T08:11:34.159 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../53-python3-repoze.lru_0.7-2_all.deb ... 2024-02-10T08:11:34.203 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-02-10T08:11:34.420 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-tempita. 2024-02-10T08:11:34.433 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../36-python3-tempita_0.5.2-6_all.deb ... 2024-02-10T08:11:35.012 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-02-10T08:11:35.206 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjs-underscore. 2024-02-10T08:11:35.220 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../54-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-02-10T08:11:35.255 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-02-10T08:11:35.456 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-paste. 2024-02-10T08:11:35.471 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../37-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-02-10T08:11:35.508 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-02-10T08:11:35.514 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-02-10T08:11:35.516 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../55-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-02-10T08:11:35.557 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-02-10T08:11:35.834 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-02-10T08:11:35.848 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../38-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-02-10T08:11:35.860 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-routes. 2024-02-10T08:11:35.873 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../56-python3-routes_2.4.1-2_all.deb ... 2024-02-10T08:11:35.892 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-02-10T08:11:35.918 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-02-10T08:11:36.195 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pastedeploy. 2024-02-10T08:11:36.211 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../39-python3-pastedeploy_2.1.0-1_all.deb ... 2024-02-10T08:11:36.253 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-02-10T08:11:36.254 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-02-10T08:11:36.263 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../57-ceph-mgr-dashboard_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:36.304 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:36.556 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-webtest. 2024-02-10T08:11:36.571 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../40-python3-webtest_2.0.32-2_all.deb ... 2024-02-10T08:11:36.614 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-02-10T08:11:36.917 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pecan. 2024-02-10T08:11:36.931 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../41-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-02-10T08:11:36.975 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-02-10T08:11:37.313 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-decorator. 2024-02-10T08:11:37.327 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../58-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-02-10T08:11:37.330 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-werkzeug. 2024-02-10T08:11:37.344 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../42-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2024-02-10T08:11:37.379 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-02-10T08:11:37.388 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-02-10T08:11:37.724 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-02-10T08:11:37.738 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../59-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-02-10T08:11:37.782 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-02-10T08:11:37.783 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-02-10T08:11:37.799 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../43-libsqlite3-mod-ceph_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:37.841 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:38.104 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-scipy. 2024-02-10T08:11:38.116 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../60-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-02-10T08:11:38.160 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-02-10T08:11:38.178 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mgr. 2024-02-10T08:11:38.191 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../44-ceph-mgr_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:38.236 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mgr (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:38.723 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mon. 2024-02-10T08:11:38.736 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../45-ceph-mon_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:38.780 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mon (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:39.435 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-02-10T08:11:39.451 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../61-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-02-10T08:11:39.484 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-02-10T08:11:39.536 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-osd. 2024-02-10T08:11:39.548 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../46-ceph-osd_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:39.592 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-osd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:40.030 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-joblib. 2024-02-10T08:11:40.043 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../62-python3-joblib_0.14.0-3_all.deb ... 2024-02-10T08:11:40.087 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-02-10T08:11:40.470 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-sklearn. 2024-02-10T08:11:40.484 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../63-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-02-10T08:11:40.528 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-02-10T08:11:41.204 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-02-10T08:11:41.219 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../64-ceph-mgr-diskprediction-local_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:41.262 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:41.581 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph. 2024-02-10T08:11:41.596 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../47-ceph_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:41.639 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:41.958 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-fuse. 2024-02-10T08:11:41.972 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../48-ceph-fuse_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:42.016 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:42.364 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cachetools. 2024-02-10T08:11:42.373 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../65-python3-cachetools_4.0.0-1_all.deb ... 2024-02-10T08:11:42.414 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-02-10T08:11:42.420 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mds. 2024-02-10T08:11:42.435 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../49-ceph-mds_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:42.478 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mds (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:42.717 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyasn1. 2024-02-10T08:11:42.725 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../66-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-02-10T08:11:42.766 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-02-10T08:11:42.981 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package cephadm. 2024-02-10T08:11:42.994 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../50-cephadm_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:43.030 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:43.095 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-02-10T08:11:43.111 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../67-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-02-10T08:11:43.153 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-02-10T08:11:43.358 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-jinja2. 2024-02-10T08:11:43.367 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../51-python3-jinja2_2.10.1-2ubuntu0.2_all.deb ... 2024-02-10T08:11:43.408 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-02-10T08:11:43.448 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rsa. 2024-02-10T08:11:43.463 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../68-python3-rsa_4.0-3_all.deb ... 2024-02-10T08:11:43.506 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rsa (4.0-3) ... 2024-02-10T08:11:43.728 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-02-10T08:11:43.741 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../52-ceph-mgr-cephadm_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:43.786 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:43.817 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-google-auth. 2024-02-10T08:11:43.826 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../69-python3-google-auth_1.5.1-2_all.deb ... 2024-02-10T08:11:43.867 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-02-10T08:11:44.122 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-repoze.lru. 2024-02-10T08:11:44.136 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../53-python3-repoze.lru_0.7-2_all.deb ... 2024-02-10T08:11:44.180 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-02-10T08:11:44.190 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-blinker. 2024-02-10T08:11:44.204 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../70-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-02-10T08:11:44.248 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-02-10T08:11:44.433 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libjs-underscore. 2024-02-10T08:11:44.446 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../54-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-02-10T08:11:44.491 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-02-10T08:11:44.559 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-oauthlib. 2024-02-10T08:11:44.575 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../71-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-02-10T08:11:44.618 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-02-10T08:11:44.760 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-02-10T08:11:44.774 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../55-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-02-10T08:11:44.810 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-02-10T08:11:44.928 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-02-10T08:11:44.944 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../72-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-02-10T08:11:44.986 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-02-10T08:11:45.096 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-routes. 2024-02-10T08:11:45.111 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../56-python3-routes_2.4.1-2_all.deb ... 2024-02-10T08:11:45.154 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-02-10T08:11:45.322 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-websocket. 2024-02-10T08:11:45.337 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../73-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-02-10T08:11:45.380 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-02-10T08:11:45.482 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-02-10T08:11:45.495 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../57-ceph-mgr-dashboard_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:45.540 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:45.691 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-kubernetes. 2024-02-10T08:11:45.707 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../74-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-02-10T08:11:45.749 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-02-10T08:11:46.305 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-02-10T08:11:46.320 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../75-ceph-mgr-k8sevents_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:46.363 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:46.647 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-decorator. 2024-02-10T08:11:46.663 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../58-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-02-10T08:11:46.665 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libonig5:amd64. 2024-02-10T08:11:46.679 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../76-libonig5_6.9.4-1_amd64.deb ... 2024-02-10T08:11:46.705 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-02-10T08:11:46.723 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-02-10T08:11:47.033 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-02-10T08:11:47.046 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../59-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-02-10T08:11:47.076 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjq1:amd64. 2024-02-10T08:11:47.090 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../77-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-02-10T08:11:47.091 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-02-10T08:11:47.134 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:11:47.394 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-scipy. 2024-02-10T08:11:47.408 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../60-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-02-10T08:11:47.412 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package jq. 2024-02-10T08:11:47.426 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../78-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-02-10T08:11:47.443 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-02-10T08:11:47.470 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:11:47.722 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package socat. 2024-02-10T08:11:47.736 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../79-socat_1.7.3.3-2_amd64.deb ... 2024-02-10T08:11:47.780 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking socat (1.7.3.3-2) ... 2024-02-10T08:11:48.100 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package xmlstarlet. 2024-02-10T08:11:48.115 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../80-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-02-10T08:11:48.158 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-02-10T08:11:48.581 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-test. 2024-02-10T08:11:48.595 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../81-ceph-test_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:48.639 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-test (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:48.782 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-02-10T08:11:48.796 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../61-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-02-10T08:11:48.840 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-02-10T08:11:49.402 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-joblib. 2024-02-10T08:11:49.416 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../62-python3-joblib_0.14.0-3_all.deb ... 2024-02-10T08:11:49.460 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-02-10T08:11:49.847 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-sklearn. 2024-02-10T08:11:49.863 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../63-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-02-10T08:11:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-02-10T08:11:50.807 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-02-10T08:11:50.823 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../64-ceph-mgr-diskprediction-local_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:50.999 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:52.120 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-cachetools. 2024-02-10T08:11:52.136 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../65-python3-cachetools_4.0.0-1_all.deb ... 2024-02-10T08:11:52.170 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-02-10T08:11:52.452 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libcephfs-dev. 2024-02-10T08:11:52.456 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pyasn1. 2024-02-10T08:11:52.465 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../66-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-02-10T08:11:52.467 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../82-libcephfs-dev_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:52.506 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-02-10T08:11:52.510 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libcephfs-dev (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:52.841 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-02-10T08:11:52.850 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../67-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-02-10T08:11:52.890 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-02-10T08:11:52.905 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package nvme-cli. 2024-02-10T08:11:52.920 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../83-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-02-10T08:11:53.012 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-02-10T08:11:53.185 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-rsa. 2024-02-10T08:11:53.199 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../68-python3-rsa_4.0-3_all.deb ... 2024-02-10T08:11:53.243 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-rsa (4.0-3) ... 2024-02-10T08:11:53.429 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-atomicwrites. 2024-02-10T08:11:53.443 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../84-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-02-10T08:11:53.496 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-02-10T08:11:53.555 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-google-auth. 2024-02-10T08:11:53.570 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../69-python3-google-auth_1.5.1-2_all.deb ... 2024-02-10T08:11:53.613 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-02-10T08:11:53.806 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-attr. 2024-02-10T08:11:53.821 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../85-python3-attr_19.3.0-2_all.deb ... 2024-02-10T08:11:53.865 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-02-10T08:11:53.933 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-blinker. 2024-02-10T08:11:53.947 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../70-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-02-10T08:11:53.992 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-02-10T08:11:54.201 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webencodings. 2024-02-10T08:11:54.214 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../86-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-02-10T08:11:54.259 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-02-10T08:11:54.286 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-oauthlib. 2024-02-10T08:11:54.300 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../71-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-02-10T08:11:54.344 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-02-10T08:11:54.603 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-html5lib. 2024-02-10T08:11:54.620 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../87-python3-html5lib_1.0.1-2_all.deb ... 2024-02-10T08:11:54.670 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-02-10T08:11:54.672 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-02-10T08:11:54.687 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../72-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-02-10T08:11:54.730 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-02-10T08:11:55.041 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-websocket. 2024-02-10T08:11:55.055 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../73-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-02-10T08:11:55.073 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-02-10T08:11:55.088 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../88-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-02-10T08:11:55.099 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-02-10T08:11:55.131 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-02-10T08:11:55.402 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-kubernetes. 2024-02-10T08:11:55.416 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../74-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-02-10T08:11:55.460 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-02-10T08:11:55.584 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-packaging. 2024-02-10T08:11:55.599 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../89-python3-packaging_20.3-1_all.deb ... 2024-02-10T08:11:55.642 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-packaging (20.3-1) ... 2024-02-10T08:11:55.928 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pastescript. 2024-02-10T08:11:55.943 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../90-python3-pastescript_2.0.2-4_all.deb ... 2024-02-10T08:11:55.994 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-02-10T08:11:56.040 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-02-10T08:11:56.055 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../75-ceph-mgr-k8sevents_16.2.14-521-g0e714d9a-1focal_all.deb ... 2024-02-10T08:11:56.098 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:56.373 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pluggy. 2024-02-10T08:11:56.387 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../91-python3-pluggy_0.13.0-2_all.deb ... 2024-02-10T08:11:56.418 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libonig5:amd64. 2024-02-10T08:11:56.432 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../76-libonig5_6.9.4-1_amd64.deb ... 2024-02-10T08:11:56.439 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-02-10T08:11:56.476 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-02-10T08:11:56.834 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-psutil. 2024-02-10T08:11:56.837 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libjq1:amd64. 2024-02-10T08:11:56.848 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../92-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-02-10T08:11:56.852 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../77-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-02-10T08:11:56.892 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-02-10T08:11:56.895 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:11:57.164 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package jq. 2024-02-10T08:11:57.180 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../78-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-02-10T08:11:57.222 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:11:57.278 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-py. 2024-02-10T08:11:57.292 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../93-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-02-10T08:11:57.344 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-02-10T08:11:57.475 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package socat. 2024-02-10T08:11:57.488 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../79-socat_1.7.3.3-2_amd64.deb ... 2024-02-10T08:11:57.533 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking socat (1.7.3.3-2) ... 2024-02-10T08:11:57.664 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyinotify. 2024-02-10T08:11:57.678 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../94-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-02-10T08:11:57.722 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-02-10T08:11:57.861 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package xmlstarlet. 2024-02-10T08:11:57.875 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../80-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-02-10T08:11:57.919 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-02-10T08:11:58.016 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-wcwidth. 2024-02-10T08:11:58.031 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../95-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-02-10T08:11:58.074 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-02-10T08:11:58.369 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pytest. 2024-02-10T08:11:58.377 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package ceph-test. 2024-02-10T08:11:58.378 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../96-python3-pytest_4.6.9-1_all.deb ... 2024-02-10T08:11:58.391 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../81-ceph-test_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:58.418 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-02-10T08:11:58.435 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking ceph-test (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:11:58.764 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-simplejson. 2024-02-10T08:11:58.778 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../97-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-02-10T08:11:58.822 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-02-10T08:11:59.241 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package radosgw. 2024-02-10T08:11:59.255 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../98-radosgw_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:11:59.299 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking radosgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:00.479 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package rbd-fuse. 2024-02-10T08:12:00.487 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../99-rbd-fuse_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:12:00.528 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking rbd-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:00.847 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-attr (19.3.0-2) ... 2024-02-10T08:12:01.150 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2024-02-10T08:12:01.275 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-02-10T08:12:01.393 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-02-10T08:12:01.594 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:02.165 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user cephadm....done 2024-02-10T08:12:02.222 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package libcephfs-dev. 2024-02-10T08:12:02.231 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../82-libcephfs-dev_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:12:02.272 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking libcephfs-dev (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:02.317 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-02-10T08:12:02.667 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package nvme-cli. 2024-02-10T08:12:02.668 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-02-10T08:12:02.676 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../83-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-02-10T08:12:02.732 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-02-10T08:12:02.953 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-02-10T08:12:03.081 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-02-10T08:12:03.157 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-atomicwrites. 2024-02-10T08:12:03.172 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../84-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-02-10T08:12:03.223 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-02-10T08:12:03.384 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-02-10T08:12:03.575 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-attr. 2024-02-10T08:12:03.585 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../85-python3-attr_19.3.0-2_all.deb ... 2024-02-10T08:12:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-02-10T08:12:03.786 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-02-10T08:12:03.970 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-webencodings. 2024-02-10T08:12:03.979 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../86-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-02-10T08:12:04.011 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-02-10T08:12:04.080 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-02-10T08:12:04.317 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-html5lib. 2024-02-10T08:12:04.324 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../87-python3-html5lib_1.0.1-2_all.deb ... 2024-02-10T08:12:04.364 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-02-10T08:12:04.372 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-02-10T08:12:04.672 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-02-10T08:12:04.736 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-02-10T08:12:04.743 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../88-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-02-10T08:12:04.783 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-02-10T08:12:04.965 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-ceph-argparse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:05.257 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-02-10T08:12:05.289 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-packaging. 2024-02-10T08:12:05.302 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../89-python3-packaging_20.3-1_all.deb ... 2024-02-10T08:12:05.345 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-packaging (20.3-1) ... 2024-02-10T08:12:05.384 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-02-10T08:12:05.676 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-02-10T08:12:05.689 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pastescript. 2024-02-10T08:12:05.704 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../90-python3-pastescript_2.0.2-4_all.deb ... 2024-02-10T08:12:05.756 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-02-10T08:12:05.977 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-02-10T08:12:06.109 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pluggy. 2024-02-10T08:12:06.123 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../91-python3-pluggy_0.13.0-2_all.deb ... 2024-02-10T08:12:06.167 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-02-10T08:12:06.370 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-02-10T08:12:06.487 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-psutil. 2024-02-10T08:12:06.502 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../92-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-02-10T08:12:06.546 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-02-10T08:12:06.647 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-02-10T08:12:06.864 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-py. 2024-02-10T08:12:06.880 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../93-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-02-10T08:12:06.922 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-02-10T08:12:06.973 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-02-10T08:12:07.243 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pyinotify. 2024-02-10T08:12:07.257 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../94-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-02-10T08:12:07.265 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjaeger (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:07.300 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-02-10T08:12:07.391 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-02-10T08:12:07.612 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-wcwidth. 2024-02-10T08:12:07.626 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../95-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-02-10T08:12:07.670 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-02-10T08:12:07.725 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-02-10T08:12:07.956 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-pytest. 2024-02-10T08:12:07.972 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../96-python3-pytest_4.6.9-1_all.deb ... 2024-02-10T08:12:08.014 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-02-10T08:12:08.021 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-packaging (20.3-1) ... 2024-02-10T08:12:08.364 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-02-10T08:12:08.384 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package python3-simplejson. 2024-02-10T08:12:08.399 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../97-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-02-10T08:12:08.450 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-02-10T08:12:08.490 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-02-10T08:12:08.774 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-02-10T08:12:08.923 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package radosgw. 2024-02-10T08:12:08.940 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../98-radosgw_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:12:08.989 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking radosgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:09.168 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-02-10T08:12:09.530 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cryptography (2.8-3ubuntu0.2) ... 2024-02-10T08:12:09.907 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up socat (1.7.3.3-2) ... 2024-02-10T08:12:10.035 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:10.193 INFO:teuthology.orchestra.run.smithi067.stdout:Selecting previously unselected package rbd-fuse. 2024-02-10T08:12:10.207 INFO:teuthology.orchestra.run.smithi067.stdout:Preparing to unpack .../99-rbd-fuse_16.2.14-521-g0e714d9a-1focal_amd64.deb ... 2024-02-10T08:12:10.251 INFO:teuthology.orchestra.run.smithi067.stdout:Unpacking rbd-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:10.344 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-02-10T08:12:10.575 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-attr (19.3.0-2) ... 2024-02-10T08:12:10.696 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-02-10T08:12:10.822 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-02-10T08:12:10.884 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2024-02-10T08:12:10.982 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-02-10T08:12:11.035 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-02-10T08:12:11.161 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-02-10T08:12:11.274 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-02-10T08:12:11.354 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:11.584 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-02-10T08:12:11.710 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-02-10T08:12:11.907 INFO:teuthology.orchestra.run.smithi067.stdout:Adding system user cephadm....done 2024-02-10T08:12:12.045 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-02-10T08:12:12.059 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-02-10T08:12:12.354 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-02-10T08:12:12.410 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-02-10T08:12:12.639 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-02-10T08:12:12.712 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-02-10T08:12:12.765 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-02-10T08:12:12.830 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-02-10T08:12:13.049 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-02-10T08:12:13.157 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-02-10T08:12:13.334 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-02-10T08:12:13.550 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-02-10T08:12:13.643 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-02-10T08:12:13.843 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-02-10T08:12:14.050 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-02-10T08:12:14.122 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-02-10T08:12:14.296 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-02-10T08:12:14.422 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-02-10T08:12:14.423 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-02-10T08:12:14.716 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-ceph-argparse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:14.740 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-02-10T08:12:15.001 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-02-10T08:12:15.099 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-02-10T08:12:15.127 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-02-10T08:12:15.225 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-02-10T08:12:15.351 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-02-10T08:12:15.411 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-02-10T08:12:15.686 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-02-10T08:12:15.706 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-02-10T08:12:16.004 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librados2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:16.079 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-02-10T08:12:16.130 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-02-10T08:12:16.382 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-02-10T08:12:16.742 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-02-10T08:12:17.026 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libjaeger (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:17.152 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.2) ... 2024-02-10T08:12:17.495 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-02-10T08:12:17.630 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librgw2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:17.756 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:17.771 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-packaging (20.3-1) ... 2024-02-10T08:12:17.882 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:12:18.008 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-02-10T08:12:18.089 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-02-10T08:12:18.214 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-02-10T08:12:18.388 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-02-10T08:12:18.499 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-02-10T08:12:18.697 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-02-10T08:12:18.900 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2024-02-10T08:12:18.990 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-02-10T08:12:19.286 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-cryptography (2.8-3ubuntu0.2) ... 2024-02-10T08:12:19.325 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libcephfs2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:19.451 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-02-10T08:12:19.645 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up socat (1.7.3.3-2) ... 2024-02-10T08:12:19.771 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:19.794 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libradosstriper1 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:19.948 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-02-10T08:12:20.089 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-02-10T08:12:20.282 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-02-10T08:12:20.440 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-02-10T08:12:20.566 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-02-10T08:12:20.692 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-02-10T08:12:20.977 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-02-10T08:12:21.294 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-02-10T08:12:21.313 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librbd1 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:21.420 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-02-10T08:12:21.439 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-02-10T08:12:21.565 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:12:21.691 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rsa (4.0-3) ... 2024-02-10T08:12:21.931 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-02-10T08:12:21.984 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-02-10T08:12:22.224 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-02-10T08:12:22.285 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:22.411 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:22.525 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-02-10T08:12:22.627 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-10T08:12:22.627 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-10T08:12:22.642 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-02-10T08:12:22.929 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-02-10T08:12:23.104 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libcephfs-dev (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:23.231 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-02-10T08:12:23.231 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-02-10T08:12:23.523 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-02-10T08:12:23.540 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-02-10T08:12:23.841 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-02-10T08:12:23.873 INFO:teuthology.orchestra.run.smithi067.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-02-10T08:12:24.101 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-02-10T08:12:24.210 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rados (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:24.226 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-02-10T08:12:24.336 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rbd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:24.478 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up rbd-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:24.544 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-02-10T08:12:24.613 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-routes (2.4.1-2) ... 2024-02-10T08:12:24.905 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-02-10T08:12:24.924 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-02-10T08:12:25.050 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-02-10T08:12:25.176 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-02-10T08:12:25.284 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:25.410 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cephfs (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:25.510 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-02-10T08:12:25.703 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-02-10T08:12:25.811 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up librados2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:25.937 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-02-10T08:12:26.649 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:27.193 INFO:teuthology.orchestra.run.smithi134.stdout:Adding group ceph....done 2024-02-10T08:12:27.486 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up librgw2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:27.612 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libsqlite3-mod-ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:27.636 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user ceph....done 2024-02-10T08:12:27.763 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:12:27.813 INFO:teuthology.orchestra.run.smithi134.stdout:Setting system user ceph properties....done 2024-02-10T08:12:27.838 INFO:teuthology.orchestra.run.smithi134.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-02-10T08:12:27.898 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-02-10T08:12:27.990 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-02-10T08:12:28.274 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-02-10T08:12:28.306 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-02-10T08:12:28.617 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-02-10T08:12:28.767 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-test (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:28.893 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up radosgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:28.918 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-02-10T08:12:29.166 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-10T08:12:29.166 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-10T08:12:29.269 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libcephfs2 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:29.395 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-02-10T08:12:29.721 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libradosstriper1 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:29.848 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-02-10T08:12:29.861 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-base (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:30.174 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-02-10T08:12:30.291 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-02-10T08:12:30.825 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mds (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:31.038 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-10T08:12:31.038 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-10T08:12:31.263 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up librbd1 (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:31.389 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-02-10T08:12:31.516 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:31.522 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-02-10T08:12:31.632 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-rsa (4.0-3) ... 2024-02-10T08:12:31.753 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-10T08:12:31.753 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-10T08:12:31.933 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-02-10T08:12:32.220 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-osd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:32.234 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mgr-modules-core (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:32.360 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:32.588 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-10T08:12:32.589 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-10T08:12:32.711 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-10T08:12:32.711 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-10T08:12:33.069 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up libcephfs-dev (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:33.177 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:33.195 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-02-10T08:12:33.286 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:33.429 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mon (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:33.488 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-02-10T08:12:33.652 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-10T08:12:33.652 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-10T08:12:33.814 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-02-10T08:12:34.147 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:34.158 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-rados (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:34.282 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:34.284 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-rbd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:34.408 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:34.410 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up rbd-fuse (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:34.536 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-routes (2.4.1-2) ... 2024-02-10T08:12:34.551 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-02-10T08:12:34.837 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-02-10T08:12:34.860 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-02-10T08:12:35.232 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-rgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:35.358 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-cephfs (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:35.659 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-02-10T08:12:36.638 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-common (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:37.191 INFO:teuthology.orchestra.run.smithi067.stdout:Adding group ceph....done 2024-02-10T08:12:37.600 INFO:teuthology.orchestra.run.smithi067.stdout:Adding system user ceph....done 2024-02-10T08:12:37.777 INFO:teuthology.orchestra.run.smithi067.stdout:Setting system user ceph properties....done 2024-02-10T08:12:37.793 INFO:teuthology.orchestra.run.smithi067.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-02-10T08:12:37.798 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-02-10T08:12:37.945 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-02-10T08:12:38.248 INFO:teuthology.orchestra.run.smithi134.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-02-10T08:12:38.254 DEBUG:teuthology.parallel:result is None 2024-02-10T08:12:38.256 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-02-10T08:12:38.722 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-test (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:38.848 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up radosgw (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:39.123 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-10T08:12:39.123 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-10T08:12:39.792 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-base (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:40.174 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-02-10T08:12:40.643 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mds (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:40.865 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-10T08:12:40.865 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-10T08:12:41.328 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mgr (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-10T08:12:41.559 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-10T08:12:42.012 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-osd (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:42.524 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-10T08:12:42.524 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-10T08:12:42.984 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mgr-k8sevents (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:43.090 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:43.224 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mon (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:43.449 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-10T08:12:43.449 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-10T08:12:43.901 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mgr-cephadm (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:44.035 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:44.161 INFO:teuthology.orchestra.run.smithi067.stdout:Setting up ceph-mgr-dashboard (16.2.14-521-g0e714d9a-1focal) ... 2024-02-10T08:12:44.304 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-02-10T08:12:44.606 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-02-10T08:12:47.587 INFO:teuthology.orchestra.run.smithi067.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-02-10T08:12:48.050 INFO:teuthology.orchestra.run.smithi067.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-02-10T08:12:48.056 DEBUG:teuthology.parallel:result is None 2024-02-10T08:12:48.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:12:48.212 DEBUG:teuthology.orchestra.run.smithi067:> dpkg-query -W -f '${Version}' ceph 2024-02-10T08:12:48.236 INFO:teuthology.orchestra.run.smithi067.stdout:16.2.14-521-g0e714d9a-1focal 2024-02-10T08:12:48.237 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-521-g0e714d9a-1focal 2024-02-10T08:12:48.237 INFO:teuthology.task.install:The correct ceph version 16.2.14-521-g0e714d9a-1focal is installed. 2024-02-10T08:12:48.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:12:48.404 DEBUG:teuthology.orchestra.run.smithi134:> dpkg-query -W -f '${Version}' ceph 2024-02-10T08:12:48.427 INFO:teuthology.orchestra.run.smithi134.stdout:16.2.14-521-g0e714d9a-1focal 2024-02-10T08:12:48.428 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-521-g0e714d9a-1focal 2024-02-10T08:12:48.428 INFO:teuthology.task.install:The correct ceph version 16.2.14-521-g0e714d9a-1focal is installed. 2024-02-10T08:12:48.429 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-10T08:12:48.430 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:12:48.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-10T08:12:48.446 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:12:48.446 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-10T08:12:48.486 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-10T08:12:48.486 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:12:48.486 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-02-10T08:12:48.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-10T08:12:48.566 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:12:48.566 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-02-10T08:12:48.580 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-10T08:12:48.637 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-10T08:12:48.638 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:12:48.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-10T08:12:48.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-10T08:12:48.709 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:12:48.710 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-10T08:12:48.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-10T08:12:48.779 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-10T08:12:48.780 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:12:48.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-02-10T08:12:48.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-10T08:12:48.853 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:12:48.854 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-02-10T08:12:48.868 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-10T08:12:48.925 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-10T08:12:49.003 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(MON_DOWN\\)', 'missing hit_sets', 'do not have an application enabled'], 'sha1': '0e714d9a4bd2a821113e6318adb87bd06cf81ec1', 'cephadm_mode': 'cephadm-package'} 2024-02-10T08:12:49.003 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:12:49.003 INFO:tasks.cephadm:Cluster fsid is 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:12:49.004 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-10T08:12:49.004 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.c': '[v2:172.21.15.67:3301,v1:172.21.15.67:6790]', 'mon.b': '172.21.15.134'} 2024-02-10T08:12:49.004 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-02-10T08:12:49.004 INFO:tasks.cephadm:First mgr is y 2024-02-10T08:12:49.004 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-10T08:12:49.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-02-10T08:12:49.021 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-02-10T08:12:49.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-02-10T08:12:49.076 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-02-10T08:12:49.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-02-10T08:12:49.134 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-02-10T08:12:49.155 INFO:tasks.cephadm:Writing seed config... 2024-02-10T08:12:49.156 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-10T08:12:49.156 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-10T08:12:49.156 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-10T08:12:49.157 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-10T08:12:49.158 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:12:49.158 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-10T08:12:49.185 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2dd26d0c-c7ec-11ee-95b8-87774f69a715 mon election default strategy = 1 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-10T08:12:49.185 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a.service 2024-02-10T08:12:49.228 DEBUG:teuthology.orchestra.run.smithi067:mgr.y> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.y.service 2024-02-10T08:12:49.271 INFO:tasks.cephadm:Bootstrapping... 2024-02-10T08:12:49.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 -v bootstrap --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:12:49.675 INFO:journalctl@ceph.mgr.y.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:12:49.682 INFO:journalctl@ceph.mon.a.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:12:49.741 INFO:teuthology.orchestra.run.smithi067.stderr:-------------------------------------------------------------------------------- 2024-02-10T08:12:49.742 INFO:teuthology.orchestra.run.smithi067.stderr:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1', '-v', 'bootstrap', '--fsid', '2dd26d0c-c7ec-11ee-95b8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-02-10T08:12:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:Verifying podman|docker is present... 2024-02-10T08:12:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:Verifying lvm2 is present... 2024-02-10T08:12:49.743 INFO:teuthology.orchestra.run.smithi067.stderr:Verifying time synchronization is in place... 2024-02-10T08:12:49.749 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T08:12:49.757 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.765 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-02-10T08:12:49.771 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.778 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: masked 2024-02-10T08:12:49.785 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.793 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-02-10T08:12:49.800 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.809 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: enabled 2024-02-10T08:12:49.816 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: active 2024-02-10T08:12:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:Unit ntp.service is enabled and running 2024-02-10T08:12:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:Repeating the final host check... 2024-02-10T08:12:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:docker (/usr/bin/docker) is present 2024-02-10T08:12:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl is present 2024-02-10T08:12:49.818 INFO:teuthology.orchestra.run.smithi067.stderr:lvcreate is present 2024-02-10T08:12:49.824 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T08:12:49.830 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.837 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-02-10T08:12:49.844 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.850 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: masked 2024-02-10T08:12:49.857 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.865 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-02-10T08:12:49.872 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: inactive 2024-02-10T08:12:49.879 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: enabled 2024-02-10T08:12:49.885 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: active 2024-02-10T08:12:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:Unit ntp.service is enabled and running 2024-02-10T08:12:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:Host looks OK 2024-02-10T08:12:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:Cluster fsid: 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:12:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:Acquiring lock 140350781709040 on /run/cephadm/2dd26d0c-c7ec-11ee-95b8-87774f69a715.lock 2024-02-10T08:12:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:Lock 140350781709040 acquired on /run/cephadm/2dd26d0c-c7ec-11ee-95b8-87774f69a715.lock 2024-02-10T08:12:49.888 INFO:teuthology.orchestra.run.smithi067.stderr:Verifying IP 172.21.15.67 port 3300 ... 2024-02-10T08:12:49.888 INFO:teuthology.orchestra.run.smithi067.stderr:Verifying IP 172.21.15.67 port 6789 ... 2024-02-10T08:12:49.888 INFO:teuthology.orchestra.run.smithi067.stderr:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-02-10T08:12:49.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: default via 172.21.15.254 dev enp3s0f1 2024-02-10T08:12:49.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.67 metric 100 2024-02-10T08:12:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-02-10T08:12:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 2024-02-10T08:12:49.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.67 metric 100 2024-02-10T08:12:49.898 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-02-10T08:12:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-10T08:12:49.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1794sec pref medium 2024-02-10T08:12:49.905 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-10T08:12:49.905 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: inet6 ::1/128 scope host 2024-02-10T08:12:49.905 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: valid_lft forever preferred_lft forever 2024-02-10T08:12:49.905 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-10T08:12:49.905 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/sbin/ip: valid_lft forever preferred_lft forever 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:Mon IP `172.21.15.67` is in CIDR network `172.21.15.254/32` 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:Mon IP `172.21.15.67` is in CIDR network `172.21.15.254/32` 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.15.254/32', '172.21.15.254/32'] 2024-02-10T08:12:49.906 INFO:teuthology.orchestra.run.smithi067.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-10T08:12:49.907 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1... 2024-02-10T08:12:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 0e714d9a4bd2a821113e6318adb87bd06cf81ec1: Pulling from ceph-ci/ceph 2024-02-10T08:12:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 58df4e03ba06: Pulling fs layer 2024-02-10T08:12:50.305 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 955bfa9dba4a: Pulling fs layer 2024-02-10T08:12:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 58df4e03ba06: Verifying Checksum 2024-02-10T08:12:59.116 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 58df4e03ba06: Download complete 2024-02-10T08:13:03.101 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 58df4e03ba06: Pull complete 2024-02-10T08:13:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 955bfa9dba4a: Verifying Checksum 2024-02-10T08:13:09.390 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 955bfa9dba4a: Download complete 2024-02-10T08:13:22.062 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: 955bfa9dba4a: Pull complete 2024-02-10T08:13:22.189 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: Digest: sha256:a9c1a3337ae6b9328bb9087151b372dee3bd15abdc1220ab67654538650c50dc 2024-02-10T08:13:22.238 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: Status: Downloaded newer image for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:13:22.264 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/docker: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 2024-02-10T08:13:33.694 INFO:teuthology.orchestra.run.smithi067.stderr:ceph: ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable) 2024-02-10T08:13:34.318 INFO:teuthology.orchestra.run.smithi067.stderr:Ceph version: ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable) 2024-02-10T08:13:34.318 INFO:teuthology.orchestra.run.smithi067.stderr:Extracting ceph user uid/gid from container image... 2024-02-10T08:13:35.120 INFO:teuthology.orchestra.run.smithi067.stderr:stat: 167 167 2024-02-10T08:13:35.794 INFO:teuthology.orchestra.run.smithi067.stderr:Creating initial keys... 2024-02-10T08:13:36.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-authtool: AQAwMMdlN4yKIhAAo+kQ7f997cXmu4PAhbM6ow== 2024-02-10T08:13:38.124 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-authtool: AQAyMMdl7yNlBxAAJhWGPVFmjV7kgr5pPXal2w== 2024-02-10T08:13:39.500 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-authtool: AQAzMMdlf2/THRAA9XhQs3T7kCwGJMl0wJ0OMw== 2024-02-10T08:13:40.235 INFO:teuthology.orchestra.run.smithi067.stderr:Creating initial monmap... 2024-02-10T08:13:41.042 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T08:13:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:41.043 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T08:13:41.664 INFO:teuthology.orchestra.run.smithi067.stderr:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T08:13:41.664 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/monmaptool: set fsid to 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:41.664 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T08:13:41.664 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:13:41.664 INFO:teuthology.orchestra.run.smithi067.stderr:Creating mon... 2024-02-10T08:13:42.514 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-10T08:13:42.514 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 1 imported monmap: 2024-02-10T08:13:42.514 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: last_changed 2024-02-10T08:13:41.042140+0000 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: created 2024-02-10T08:13:41.042140+0000 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.a 2024-02-10T08:13:42.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 0 /usr/bin/ceph-mon: set fsid to 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: RocksDB version: 6.8.1 2024-02-10T08:13:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.517 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Compile date Feb 8 2024 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: DB SUMMARY 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.error_if_exists: 0 2024-02-10T08:13:42.518 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.create_if_missing: 1 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.paranoid_checks: 1 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.env: 0x55fd81026080 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.fs: Posix File System 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.info_log: 0x55fd81f3fa80 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.statistics: (nil) 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.use_fsync: 0 2024-02-10T08:13:42.545 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_log_file_size: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.allow_fallocate: 1 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.use_direct_reads: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T08:13:42.546 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.db_log_dir: 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_subcompactions: 1 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_background_flushes: -1 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T08:13:42.547 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.write_buffer_manager: 0x55fd81f45950 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-10T08:13:42.548 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.unordered_write: 0 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.row_cache: None 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.wal_filter: None 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-10T08:13:42.549 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.preserve_deletes: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.two_write_queues: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.atomic_flush: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.log_readahead_size: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_background_jobs: 2 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_background_compactions: -1 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T08:13:42.550 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.max_open_files: -1 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Compression algorithms supported: 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kZSTD supported: 0 2024-02-10T08:13:42.551 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kXpressCompression supported: 0 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kLZ4Compression supported: 1 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kBZip2Compression supported: 0 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kZlibCompression supported: 1 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: kSnappyCompression supported: 1 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.508+0000 7f34349c1880 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.552 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.merge_operator: 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_filter: None 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fd81e53d30) 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-10T08:13:42.553 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-10T08:13:42.574 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_cache: 0x55fd81e8ad10 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-10T08:13:42.575 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-10T08:13:42.576 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-10T08:13:42.577 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression: NoCompression 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.num_levels: 7 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T08:13:42.578 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T08:13:42.579 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T08:13:42.580 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T08:13:42.581 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.table_properties_collectors: 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.inplace_update_support: 0 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T08:13:42.582 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.bloom_locality: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.max_successive_merges: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.ttl: 2592000 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T08:13:42.583 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f34349c1880 4 rocksdb: DB pointer 0x55fd81f53800 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f341d833700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.512+0000 7f341d833700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T08:13:42.584 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-10T08:13:42.585 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T08:13:42.586 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-10T08:13:42.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T08:13:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.516+0000 7f34349c1880 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.516+0000 7f34349c1880 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-10T08:13:42.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph-mon: debug 2024-02-10T08:13:42.516+0000 7f34349c1880 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-02-10T08:13:43.152 INFO:teuthology.orchestra.run.smithi067.stderr:create mon.a on 2024-02-10T08:13:43.173 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-10T08:13:43.384 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-02-10T08:13:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715.target -> /etc/systemd/system/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715.target. 2024-02-10T08:13:43.585 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715.target -> /etc/systemd/system/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715.target. 2024-02-10T08:13:44.011 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to reset failed state of unit ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a.service: Unit ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a.service not loaded. 2024-02-10T08:13:44.019 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Created symlink /etc/systemd/system/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715.target.wants/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a.service -> /etc/systemd/system/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@.service. 2024-02-10T08:13:44.241 INFO:teuthology.orchestra.run.smithi067.stderr:firewalld does not appear to be present 2024-02-10T08:13:44.241 INFO:teuthology.orchestra.run.smithi067.stderr:Not possible to enable service . firewalld.service is not available 2024-02-10T08:13:44.241 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for mon to start... 2024-02-10T08:13:44.241 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for mon... 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.496+0000 7fde03947700 1 Processor -- start 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.496+0000 7fde03947700 1 -- start start 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.496+0000 7fde03947700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc0fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.496+0000 7fde03947700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc0fbfd0 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde016e3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc0fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde016e3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc0fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50068/0 (socket says 172.21.15.67:50068) 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde016e3700 1 -- 172.21.15.67:0/3344100277 learned_addr learned my addr 172.21.15.67:0/3344100277 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde016e3700 1 -- 172.21.15.67:0/3344100277 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0fc110 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.505 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde016e3700 1 --2- 172.21.15.67:0/3344100277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc0fbae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fddec009730 tx=0x7fddec0090f0).ready entity=mon.0 client_cookie=b1a403b1474fa937 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fddfbfff700 1 -- 172.21.15.67:0/3344100277 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddec00b580 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fddfbfff700 1 -- 172.21.15.67:0/3344100277 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec00b6e0 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fddfbfff700 1 -- 172.21.15.67:0/3344100277 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddec00b990 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 -- 172.21.15.67:0/3344100277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 msgr2=0x7fddfc0fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 --2- 172.21.15.67:0/3344100277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc0fbae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fddec009730 tx=0x7fddec0090f0).stop 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 -- 172.21.15.67:0/3344100277 shutdown_connections 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 --2- 172.21.15.67:0/3344100277 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc0fbae0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 -- 172.21.15.67:0/3344100277 >> 172.21.15.67:0/3344100277 conn(0x7fddfc0f6ec0 msgr2=0x7fddfc0f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 -- 172.21.15.67:0/3344100277 shutdown_connections 2024-02-10T08:13:45.506 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 -- 172.21.15.67:0/3344100277 wait complete. 2024-02-10T08:13:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 Processor -- start 2024-02-10T08:13:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.500+0000 7fde03947700 1 -- start start 2024-02-10T08:13:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde03947700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde03947700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddfc10a090 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.509 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde016e3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde016e3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50084/0 (socket says 172.21.15.67:50084) 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde016e3700 1 -- 172.21.15.67:0/3648642668 learned_addr learned my addr 172.21.15.67:0/3648642668 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde016e3700 1 -- 172.21.15.67:0/3648642668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec009370 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde016e3700 1 --2- 172.21.15.67:0/3648642668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc109ba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fddec009130 tx=0x7fddec017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddec011cd0 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec011e30 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde03947700 1 -- 172.21.15.67:0/3648642668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc10a290 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fddec01d570 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fde03947700 1 -- 172.21.15.67:0/3648642668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddfc10a720 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fddec01d930 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.510 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fddec026800 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.511 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.504+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc004f20 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.515 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.512+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fddec021030 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:45 smithi067 bash[16856]: cluster 2024-02-10T08:13:45.459900+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T08:13:45.648 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdddc0054a0 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fddfa7fc700 1 -- 172.21.15.67:0/3648642668 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fddec0269c0 con 0x7fddfc0fb6c0 2024-02-10T08:13:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: cluster: 2024-02-10T08:13:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: id: 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-10T08:13:45.649 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: services: 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.181871s) 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: data: 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-10T08:13:45.651 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: pgs: 2024-02-10T08:13:45.652 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 msgr2=0x7fddfc109ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 --2- 172.21.15.67:0/3648642668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc109ba0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fddec009130 tx=0x7fddec017000).stop 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 shutdown_connections 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 --2- 172.21.15.67:0/3648642668 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fddfc0fb6c0 0x7fddfc109ba0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 >> 172.21.15.67:0/3648642668 conn(0x7fddfc0f6ec0 msgr2=0x7fddfc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 shutdown_connections 2024-02-10T08:13:45.653 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:45.644+0000 7fdde3fff700 1 -- 172.21.15.67:0/3648642668 wait complete. 2024-02-10T08:13:46.286 INFO:teuthology.orchestra.run.smithi067.stderr:mon is available 2024-02-10T08:13:46.286 INFO:teuthology.orchestra.run.smithi067.stderr:Assimilating anything we can from ceph.conf... 2024-02-10T08:13:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:46 smithi067 bash[16856]: cluster 2024-02-10T08:13:45.466428+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T08:13:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:46 smithi067 bash[16856]: cluster 2024-02-10T08:13:45.466674+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} 2024-02-10T08:13:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:46 smithi067 bash[16856]: cluster 2024-02-10T08:13:45.492694+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-02-10T08:13:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:46 smithi067 bash[16856]: cluster 2024-02-10T08:13:45.497197+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-10T08:13:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:46 smithi067 bash[16856]: cluster 2024-02-10T08:13:45.498000+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-10T08:13:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:46 smithi067 bash[16856]: audit 2024-02-10T08:13:45.648249+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.67:0/3648642668' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-10T08:13:47.244 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb57c5a700 1 Processor -- start 2024-02-10T08:13:47.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb57c5a700 1 -- start start 2024-02-10T08:13:47.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb57c5a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb500fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:47.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb57c5a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500fe270 con 0x7ffb500fd8d0 2024-02-10T08:13:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb559f6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb500fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb559f6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb500fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50092/0 (socket says 172.21.15.67:50092) 2024-02-10T08:13:47.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb559f6700 1 -- 172.21.15.67:0/4270571855 learned_addr learned my addr 172.21.15.67:0/4270571855 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb559f6700 1 -- 172.21.15.67:0/4270571855 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500fe3b0 con 0x7ffb500fd8d0 2024-02-10T08:13:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.240+0000 7ffb559f6700 1 --2- 172.21.15.67:0/4270571855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb500fdcf0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7ffb4c009730 tx=0x7ffb4c0090f0).ready entity=mon.0 client_cookie=d0bb40b4a8407845 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:47.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb549f4700 1 -- 172.21.15.67:0/4270571855 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb4c00b580 con 0x7ffb500fd8d0 2024-02-10T08:13:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb549f4700 1 -- 172.21.15.67:0/4270571855 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c00b6e0 con 0x7ffb500fd8d0 2024-02-10T08:13:47.248 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb549f4700 1 -- 172.21.15.67:0/4270571855 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb4c00b990 con 0x7ffb500fd8d0 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- 172.21.15.67:0/4270571855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 msgr2=0x7ffb500fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 --2- 172.21.15.67:0/4270571855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb500fdcf0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7ffb4c009730 tx=0x7ffb4c0090f0).stop 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- 172.21.15.67:0/4270571855 shutdown_connections 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 --2- 172.21.15.67:0/4270571855 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb500fdcf0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- 172.21.15.67:0/4270571855 >> 172.21.15.67:0/4270571855 conn(0x7ffb50070e40 msgr2=0x7ffb50071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- 172.21.15.67:0/4270571855 shutdown_connections 2024-02-10T08:13:47.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- 172.21.15.67:0/4270571855 wait complete. 2024-02-10T08:13:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 Processor -- start 2024-02-10T08:13:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- start start 2024-02-10T08:13:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb5010be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:47.250 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb57c5a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5010c330 con 0x7ffb500fd8d0 2024-02-10T08:13:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb559f6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb5010be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb559f6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb5010be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50094/0 (socket says 172.21.15.67:50094) 2024-02-10T08:13:47.251 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.244+0000 7ffb559f6700 1 -- 172.21.15.67:0/927486232 learned_addr learned my addr 172.21.15.67:0/927486232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb559f6700 1 -- 172.21.15.67:0/927486232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c009370 con 0x7ffb500fd8d0 2024-02-10T08:13:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb559f6700 1 --2- 172.21.15.67:0/927486232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb5010be40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7ffb4c009130 tx=0x7ffb4c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb4c011cd0 con 0x7ffb500fd8d0 2024-02-10T08:13:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c011e30 con 0x7ffb500fd8d0 2024-02-10T08:13:47.252 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5010c530 con 0x7ffb500fd8d0 2024-02-10T08:13:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb4c01d570 con 0x7ffb500fd8d0 2024-02-10T08:13:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5010d140 con 0x7ffb500fd8d0 2024-02-10T08:13:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7ffb4c01da00 con 0x7ffb500fd8d0 2024-02-10T08:13:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7ffb4c026720 con 0x7ffb500fd8d0 2024-02-10T08:13:47.253 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.248+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5010c9c0 con 0x7ffb500fd8d0 2024-02-10T08:13:47.258 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.252+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7ffb4c021030 con 0x7ffb500fd8d0 2024-02-10T08:13:47.387 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.384+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ffb5005a3b0 con 0x7ffb500fd8d0 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.388+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 7 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ffb4c01de90 con 0x7ffb500fd8d0 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.388+0000 7ffb46ffd700 1 -- 172.21.15.67:0/927486232 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+363 (secure 0 0 0) 0x7ffb4c026960 con 0x7ffb500fd8d0 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: [global] 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: fsid = 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-02-10T08:13:47.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-10T08:13:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-10T08:13:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-10T08:13:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-10T08:13:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.395 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: [mgr] 2024-02-10T08:13:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-10T08:13:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: [osd] 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 msgr2=0x7ffb5010be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 --2- 172.21.15.67:0/927486232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb5010be40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7ffb4c009130 tx=0x7ffb4c017000).stop 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 shutdown_connections 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 --2- 172.21.15.67:0/927486232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb500fd8d0 0x7ffb5010be40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 >> 172.21.15.67:0/927486232 conn(0x7ffb50070e40 msgr2=0x7ffb501005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 shutdown_connections 2024-02-10T08:13:47.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:47.392+0000 7ffb57c5a700 1 -- 172.21.15.67:0/927486232 wait complete. 2024-02-10T08:13:47.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:47 smithi067 bash[16856]: audit 2024-02-10T08:13:47.387905+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.15.67:0/927486232' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T08:13:47.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:47 smithi067 bash[16856]: audit 2024-02-10T08:13:47.393214+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.15.67:0/927486232' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-10T08:13:48.024 INFO:teuthology.orchestra.run.smithi067.stderr:Generating new minimal ceph.conf... 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a0beb6700 1 Processor -- start 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a0beb6700 1 -- start start 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a0beb6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a040f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a0beb6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040fa350 con 0x7f1a040f99b0 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a09c52700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a040f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a09c52700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a040f9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50106/0 (socket says 172.21.15.67:50106) 2024-02-10T08:13:48.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.916+0000 7f1a09c52700 1 -- 172.21.15.67:0/3100019241 learned_addr learned my addr 172.21.15.67:0/3100019241 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:48.923 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a09c52700 1 -- 172.21.15.67:0/3100019241 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040fa490 con 0x7f1a040f99b0 2024-02-10T08:13:48.923 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a09c52700 1 --2- 172.21.15.67:0/3100019241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a040f9dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f19fc009e50 tx=0x7f19fc009840).ready entity=mon.0 client_cookie=eb019886bf86ad92 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:48.923 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a08c50700 1 -- 172.21.15.67:0/3100019241 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19fc00e070 con 0x7f1a040f99b0 2024-02-10T08:13:48.924 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a08c50700 1 -- 172.21.15.67:0/3100019241 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f19fc00be90 con 0x7f1a040f99b0 2024-02-10T08:13:48.924 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a08c50700 1 -- 172.21.15.67:0/3100019241 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19fc013d40 con 0x7f1a040f99b0 2024-02-10T08:13:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3100019241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 msgr2=0x7f1a040f9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 --2- 172.21.15.67:0/3100019241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a040f9dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f19fc009e50 tx=0x7f19fc009840).stop 2024-02-10T08:13:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3100019241 shutdown_connections 2024-02-10T08:13:48.925 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 --2- 172.21.15.67:0/3100019241 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a040f9dd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:48.926 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3100019241 >> 172.21.15.67:0/3100019241 conn(0x7f1a040f5460 msgr2=0x7f1a040f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:48.926 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3100019241 shutdown_connections 2024-02-10T08:13:48.926 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.920+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3100019241 wait complete. 2024-02-10T08:13:48.926 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 Processor -- start 2024-02-10T08:13:48.927 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 -- start start 2024-02-10T08:13:48.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a0410a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:48.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0410a790 con 0x7f1a040f99b0 2024-02-10T08:13:48.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a09c52700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a0410a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:48.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a09c52700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a0410a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50108/0 (socket says 172.21.15.67:50108) 2024-02-10T08:13:48.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a09c52700 1 -- 172.21.15.67:0/3974034762 learned_addr learned my addr 172.21.15.67:0/3974034762 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:48.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a09c52700 1 -- 172.21.15.67:0/3974034762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc009ae0 con 0x7f1a040f99b0 2024-02-10T08:13:48.929 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a09c52700 1 --2- 172.21.15.67:0/3974034762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a0410a2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f19fc014f60 tx=0x7f19fc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:48.929 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19fc00e040 con 0x7f1a040f99b0 2024-02-10T08:13:48.929 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0410a990 con 0x7f1a040f99b0 2024-02-10T08:13:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f19fc012900 con 0x7f1a040f99b0 2024-02-10T08:13:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19fc0076b0 con 0x7f1a040f99b0 2024-02-10T08:13:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0410ae20 con 0x7f1a040f99b0 2024-02-10T08:13:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f19fc003040 con 0x7f1a040f99b0 2024-02-10T08:13:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f19fc0257d0 con 0x7f1a040f99b0 2024-02-10T08:13:48.930 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.924+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a04046d80 con 0x7f1a040f99b0 2024-02-10T08:13:48.933 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:48.928+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f19fc018040 con 0x7f1a040f99b0 2024-02-10T08:13:49.062 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.056+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1a040fd520 con 0x7f1a040f99b0 2024-02-10T08:13:49.063 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a02ffd700 1 -- 172.21.15.67:0/3974034762 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f1a040fd520 con 0x7f1a040f99b0 2024-02-10T08:13:49.064 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 msgr2=0x7f1a0410a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:49.064 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 --2- 172.21.15.67:0/3974034762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a0410a2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f19fc014f60 tx=0x7f19fc009840).stop 2024-02-10T08:13:49.065 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 shutdown_connections 2024-02-10T08:13:49.065 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 --2- 172.21.15.67:0/3974034762 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1a040f99b0 0x7f1a0410a2a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:49.065 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 >> 172.21.15.67:0/3974034762 conn(0x7f1a040f5460 msgr2=0x7f1a040f6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:49.065 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 shutdown_connections 2024-02-10T08:13:49.065 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:49.060+0000 7f1a0beb6700 1 -- 172.21.15.67:0/3974034762 wait complete. 2024-02-10T08:13:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:49 smithi067 bash[16856]: audit 2024-02-10T08:13:49.062665+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.67:0/3974034762' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:13:49.603 INFO:teuthology.orchestra.run.smithi067.stderr:Restarting the monitor... 2024-02-10T08:13:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:49 smithi067 systemd[1]: Stopping Ceph mon.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:13:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:49 smithi067 bash[17246]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mon.a 2024-02-10T08:13:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:49 smithi067 bash[16856]: debug 2024-02-10T08:13:49.792+0000 7f3f0c6c7700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-10T08:13:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:49 smithi067 bash[16856]: debug 2024-02-10T08:13:49.792+0000 7f3f0c6c7700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-10T08:13:50.667 INFO:teuthology.orchestra.run.smithi067.stderr:Setting public_network to 172.21.0.0/20,172.21.15.254/32 in mon config section 2024-02-10T08:13:50.729 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:50 smithi067 bash[17281]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mon-a 2024-02-10T08:13:50.729 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:50 smithi067 bash[17345]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mon-a 2024-02-10T08:13:50.729 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:50 smithi067 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a.service: Succeeded. 2024-02-10T08:13:50.729 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:50 smithi067 systemd[1]: Stopped Ceph mon.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:13:50.729 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:50 smithi067 systemd[1]: Started Ceph mon.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:13:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7cea1700 1 Processor -- start 2024-02-10T08:13:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7cea1700 1 -- start start 2024-02-10T08:13:52.077 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7cea1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf78070220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7cea1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780720a0 con 0x7fbf78071c80 2024-02-10T08:13:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf78070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf78070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50118/0 (socket says 172.21.15.67:50118) 2024-02-10T08:13:52.078 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7659c700 1 -- 172.21.15.67:0/201622005 learned_addr learned my addr 172.21.15.67:0/201622005 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.072+0000 7fbf7659c700 1 -- 172.21.15.67:0/201622005 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78070710 con 0x7fbf78071c80 2024-02-10T08:13:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7659c700 1 --2- 172.21.15.67:0/201622005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf78070220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fbf68009730 tx=0x7fbf680090f0).ready entity=mon.0 client_cookie=2771db0c2a01213a server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7559a700 1 -- 172.21.15.67:0/201622005 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf6800b580 con 0x7fbf78071c80 2024-02-10T08:13:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7559a700 1 -- 172.21.15.67:0/201622005 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fbf6800bb50 con 0x7fbf78071c80 2024-02-10T08:13:52.079 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7559a700 1 -- 172.21.15.67:0/201622005 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf68011a70 con 0x7fbf78071c80 2024-02-10T08:13:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 -- 172.21.15.67:0/201622005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 msgr2=0x7fbf78070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:52.080 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 --2- 172.21.15.67:0/201622005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf78070220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fbf68009730 tx=0x7fbf680090f0).stop 2024-02-10T08:13:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 -- 172.21.15.67:0/201622005 shutdown_connections 2024-02-10T08:13:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 --2- 172.21.15.67:0/201622005 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf78070220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 -- 172.21.15.67:0/201622005 >> 172.21.15.67:0/201622005 conn(0x7fbf780f6ec0 msgr2=0x7fbf780f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 -- 172.21.15.67:0/201622005 shutdown_connections 2024-02-10T08:13:52.081 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 -- 172.21.15.67:0/201622005 wait complete. 2024-02-10T08:13:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 Processor -- start 2024-02-10T08:13:52.082 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.076+0000 7fbf7cea1700 1 -- start start 2024-02-10T08:13:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7cea1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf7810c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7cea1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7810c9b0 con 0x7fbf78071c80 2024-02-10T08:13:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf7810c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf7810c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50132/0 (socket says 172.21.15.67:50132) 2024-02-10T08:13:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7659c700 1 -- 172.21.15.67:0/583271581 learned_addr learned my addr 172.21.15.67:0/583271581 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7659c700 1 -- 172.21.15.67:0/583271581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf68009370 con 0x7fbf78071c80 2024-02-10T08:13:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7659c700 1 --2- 172.21.15.67:0/583271581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf7810c4c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fbf68009f00 tx=0x7fbf680090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf6801a640 con 0x7fbf78071c80 2024-02-10T08:13:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fbf6801ac10 con 0x7fbf78071c80 2024-02-10T08:13:52.084 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf68023b10 con 0x7fbf78071c80 2024-02-10T08:13:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7810cbb0 con 0x7fbf78071c80 2024-02-10T08:13:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7810d040 con 0x7fbf78071c80 2024-02-10T08:13:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fbf6802c3f0 con 0x7fbf78071c80 2024-02-10T08:13:52.085 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.080+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fbf68022730 con 0x7fbf78071c80 2024-02-10T08:13:52.086 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.084+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf78046d80 con 0x7fbf78071c80 2024-02-10T08:13:52.090 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.084+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7fbf6801e030 con 0x7fbf78071c80 2024-02-10T08:13:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.928+0000 7f94e0c6c880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-10T08:13:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.928+0000 7f94e0c6c880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-10T08:13:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.928+0000 7f94e0c6c880 0 pidfile_write: ignore empty --pid-file 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 0 load: jerasure load: lrc load: isa 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: RocksDB version: 6.8.1 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Compile date Feb 8 2024 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: DB SUMMARY 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: CURRENT file: CURRENT 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 87983 ; 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.error_if_exists: 0 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.create_if_missing: 0 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.paranoid_checks: 1 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.env: 0x55d0ff226080 2024-02-10T08:13:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.fs: Posix File System 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.info_log: 0x55d101574220 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.statistics: (nil) 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.use_fsync: 0 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_log_file_size: 0 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-10T08:13:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.allow_fallocate: 1 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.use_direct_reads: 0 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-10T08:13:52.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.db_log_dir: 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_subcompactions: 1 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_background_flushes: -1 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.write_buffer_manager: 0x55d1022683f0 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T08:13:52.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.unordered_write: 0 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.row_cache: None 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.wal_filter: None 2024-02-10T08:13:52.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.preserve_deletes: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.two_write_queues: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.atomic_flush: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.log_readahead_size: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_background_jobs: 2 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_background_compactions: -1 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T08:13:52.126 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_open_files: -1 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Compression algorithms supported: 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T08:13:52.127 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kZSTD supported: 0 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kXpressCompression supported: 0 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kLZ4Compression supported: 1 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kBZip2Compression supported: 0 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kZlibCompression supported: 1 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: kSnappyCompression supported: 1 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T08:13:52.128 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.merge_operator: 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_filter: None 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d101517d28) 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: cache_index_and_filter_blocks: 1 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: pin_top_level_index_and_filter: 1 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: index_type: 0 2024-02-10T08:13:52.129 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: data_block_index_type: 0 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: index_shortening: 1 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: data_block_hash_table_util_ratio: 0.750000 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: hash_index_allow_collision: 1 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: checksum: 1 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: no_block_cache: 0 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_cache: 0x55d10154ef10 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_cache_name: BinnedLRUCache 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_cache_options: 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: capacity : 536870912 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: num_shard_bits : 4 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: strict_capacity_limit : 0 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: high_pri_pool_ratio: 0.000 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_cache_compressed: (nil) 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: persistent_cache: (nil) 2024-02-10T08:13:52.130 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_size: 4096 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_size_deviation: 10 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_restart_interval: 16 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: index_block_restart_interval: 1 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: metadata_block_size: 4096 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: partition_filters: 0 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: use_delta_encoding: 1 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: whole_key_filtering: 1 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: verify_compression: 0 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: read_amp_bytes_per_bit: 0 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: format_version: 2 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: enable_index_compression: 1 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: block_align: 0 2024-02-10T08:13:52.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression: NoCompression 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.num_levels: 7 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T08:13:52.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T08:13:52.133 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T08:13:52.134 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-10T08:13:52.135 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T08:13:52.135 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T08:13:52.135 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-10T08:13:52.135 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-10T08:13:52.135 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T08:13:52.135 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.table_properties_collectors: 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.inplace_update_support: 0 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T08:13:52.136 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.bloom_locality: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.max_successive_merges: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.ttl: 2592000 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.932+0000 7f94e0c6c880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707552831943320, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-10T08:13:52.137 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707552831945360, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 84508, "table_properties": {"data_size": 82781, "index_size": 332, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 581, "raw_key_size": 9930, "raw_average_key_size": 47, "raw_value_size": 77086, "raw_average_value_size": 370, "num_data_blocks": 11, "num_entries": 208, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1707552831, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.940+0000 7f94e0c6c880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707552831946073, "job": 1, "event": "recovery_finished"} 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.944+0000 7f94e0c6c880 4 rocksdb: DB pointer 0x55d101617800 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.944+0000 7f94c7a5d700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.944+0000 7f94c7a5d700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** DB Stats ** 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T08:13:52.138 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** Compaction Stats [default] ** 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: L0 2/0 84.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Sum 2/0 84.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** Compaction Stats [default] ** 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T08:13:52.139 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(Keys): cumulative 0, interval 0 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Cumulative compaction: 0.00 GB write, 8.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Interval compaction: 0.00 GB write, 8.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** File Read Latency Histogram By Level [default] ** 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** Compaction Stats [default] ** 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: L0 2/0 84.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Sum 2/0 84.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** Compaction Stats [default] ** 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:13:52.140 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 65.1 0.00 0.00 1 0.001 0 0 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: AddFile(Keys): cumulative 0, interval 0 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Cumulative compaction: 0.00 GB write, 8.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: ** File Read Latency Histogram By Level [default] ** 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.944+0000 7f94e0c6c880 0 starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.944+0000 7f94e0c6c880 1 mon.a@-1(???) e1 preinit fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 0 mon.a@-1(???).mds e1 new map 2024-02-10T08:13:52.141 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 0 mon.a@-1(???).mds e1 print_map 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: e1 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: legacy client fscid: -1 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: No filesystems configured 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: audit 2024-02-10T08:13:49.062665+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.67:0/3974034762' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:51 smithi067 bash[17385]: debug 2024-02-10T08:13:51.948+0000 7f94e0c6c880 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:52 smithi067 bash[17385]: cluster 2024-02-10T08:13:51.975705+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:52 smithi067 bash[17385]: cluster 2024-02-10T08:13:51.975835+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} 2024-02-10T08:13:52.142 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:52 smithi067 bash[17385]: cluster 2024-02-10T08:13:51.976775+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-02-10T08:13:52.143 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:52 smithi067 bash[17385]: cluster 2024-02-10T08:13:51.976849+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-10T08:13:52.143 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:52 smithi067 bash[17385]: cluster 2024-02-10T08:13:51.977659+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-10T08:13:52.224 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.220+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fbf78101890 con 0x7fbf78071c80 2024-02-10T08:13:52.232 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf677fe700 1 -- 172.21.15.67:0/583271581 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf6801ad80 con 0x7fbf78071c80 2024-02-10T08:13:52.233 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 msgr2=0x7fbf7810c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:52.233 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 --2- 172.21.15.67:0/583271581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf7810c4c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7fbf68009f00 tx=0x7fbf680090f0).stop 2024-02-10T08:13:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 shutdown_connections 2024-02-10T08:13:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 --2- 172.21.15.67:0/583271581 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbf78071c80 0x7fbf7810c4c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 >> 172.21.15.67:0/583271581 conn(0x7fbf780f6ec0 msgr2=0x7fbf780f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 shutdown_connections 2024-02-10T08:13:52.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:52.228+0000 7fbf7cea1700 1 -- 172.21.15.67:0/583271581 wait complete. 2024-02-10T08:13:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-10T08:13:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:13:52.932 INFO:teuthology.orchestra.run.smithi067.stderr:Creating mgr... 2024-02-10T08:13:52.933 INFO:teuthology.orchestra.run.smithi067.stderr:Verifying port 9283 ... 2024-02-10T08:13:53.169 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Failed to reset failed state of unit ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.y.service: Unit ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.y.service not loaded. 2024-02-10T08:13:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:systemctl: Created symlink /etc/systemd/system/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715.target.wants/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.y.service -> /etc/systemd/system/ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@.service. 2024-02-10T08:13:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:firewalld does not appear to be present 2024-02-10T08:13:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:Not possible to enable service . firewalld.service is not available 2024-02-10T08:13:53.382 INFO:teuthology.orchestra.run.smithi067.stderr:firewalld does not appear to be present 2024-02-10T08:13:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-10T08:13:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for mgr to start... 2024-02-10T08:13:53.383 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for mgr... 2024-02-10T08:13:53.608 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:53 smithi067 bash[17385]: audit 2024-02-10T08:13:52.231484+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.15.67:0/583271581' entity='client.admin' 2024-02-10T08:13:54.850 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.840+0000 7f5909c2e700 1 Processor -- start 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.840+0000 7f5909c2e700 1 -- start start 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f59040fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59040fe270 con 0x7f59040fd8d0 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59037fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f59040fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59037fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f59040fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49952/0 (socket says 172.21.15.67:49952) 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59037fe700 1 -- 172.21.15.67:0/3348610084 learned_addr learned my addr 172.21.15.67:0/3348610084 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:54.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59037fe700 1 -- 172.21.15.67:0/3348610084 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040fe3b0 con 0x7f59040fd8d0 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59037fe700 1 --2- 172.21.15.67:0/3348610084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f59040fdcf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f58f4009720 tx=0x7f58f40090f0).ready entity=mon.0 client_cookie=44afd5fbd554da40 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59027fc700 1 -- 172.21.15.67:0/3348610084 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f400b520 con 0x7f59040fd8d0 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59027fc700 1 -- 172.21.15.67:0/3348610084 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f58f400baf0 con 0x7f59040fd8d0 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f59027fc700 1 -- 172.21.15.67:0/3348610084 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f40119f0 con 0x7f59040fd8d0 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 -- 172.21.15.67:0/3348610084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 msgr2=0x7f59040fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 --2- 172.21.15.67:0/3348610084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f59040fdcf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f58f4009720 tx=0x7f58f40090f0).stop 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 -- 172.21.15.67:0/3348610084 shutdown_connections 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 --2- 172.21.15.67:0/3348610084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f59040fdcf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 -- 172.21.15.67:0/3348610084 >> 172.21.15.67:0/3348610084 conn(0x7f59040710b0 msgr2=0x7f59040714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 -- 172.21.15.67:0/3348610084 shutdown_connections 2024-02-10T08:13:54.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.844+0000 7f5909c2e700 1 -- 172.21.15.67:0/3348610084 wait complete. 2024-02-10T08:13:54.855 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5909c2e700 1 Processor -- start 2024-02-10T08:13:54.855 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5909c2e700 1 -- start start 2024-02-10T08:13:54.855 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5909c2e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f590410c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5909c2e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590410c940 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f59037fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f590410c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f59037fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f590410c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49968/0 (socket says 172.21.15.67:49968) 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f59037fe700 1 -- 172.21.15.67:0/294463018 learned_addr learned my addr 172.21.15.67:0/294463018 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f59037fe700 1 -- 172.21.15.67:0/294463018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009310 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f59037fe700 1 --2- 172.21.15.67:0/294463018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f590410c450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f58f400bec0 tx=0x7f58f40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f401a640 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f58f401ac10 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590410cb40 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f4022af0 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590410cfd0 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f58f4023450 con 0x7f59040fd8d0 2024-02-10T08:13:54.856 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.848+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f58f4019700 con 0x7f59040fd8d0 2024-02-10T08:13:54.857 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.852+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5904046d80 con 0x7f59040fd8d0 2024-02-10T08:13:54.859 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.856+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7f58f4028060 con 0x7f59040fd8d0 2024-02-10T08:13:54.998 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.992+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f59040ff6d0 con 0x7f59040fd8d0 2024-02-10T08:13:54.998 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.992+0000 7f5900ff9700 1 -- 172.21.15.67:0/294463018 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7f58f401e070 con 0x7f59040fd8d0 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "fsid": "2dd26d0c-c7ec-11ee-95b8-87774f69a715", 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "health": { 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "checks": {}, 2024-02-10T08:13:54.999 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mutes": [] 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum": [ 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 0 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "a" 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum_age": 3, 2024-02-10T08:13:55.001 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "monmap": { 2024-02-10T08:13:55.002 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:55.002 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osdmap": { 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-10T08:13:55.007 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "pgmap": { 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.008 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "fsmap": { 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "available": false, 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "modules": [ 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "iostat", 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "nfs", 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "restful" 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "services": {} 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.009 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "servicemap": { 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "modified": "2024-02-10T08:13:45.490008+0000", 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "services": {} 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 msgr2=0x7f590410c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 --2- 172.21.15.67:0/294463018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f590410c450 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f58f400bec0 tx=0x7f58f40090f0).stop 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 shutdown_connections 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 --2- 172.21.15.67:0/294463018 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f59040fd8d0 0x7f590410c450 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 >> 172.21.15.67:0/294463018 conn(0x7f59040710b0 msgr2=0x7f59040fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 shutdown_connections 2024-02-10T08:13:55.010 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:54.996+0000 7f5909c2e700 1 -- 172.21.15.67:0/294463018 wait complete. 2024-02-10T08:13:55.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:55 smithi067 bash[17385]: audit 2024-02-10T08:13:54.998169+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.67:0/294463018' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T08:13:55.253 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:54 smithi067 bash[17647]: debug 2024-02-10T08:13:54.984+0000 7f0e4f31f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T08:13:55.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:55 smithi067 bash[17647]: debug 2024-02-10T08:13:55.476+0000 7f0e4f31f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T08:13:55.724 INFO:teuthology.orchestra.run.smithi067.stderr:mgr not available, waiting (1/15)... 2024-02-10T08:13:56.851 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:56 smithi067 bash[17647]: debug 2024-02-10T08:13:56.484+0000 7f0e4f31f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T08:13:56.851 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:56 smithi067 bash[17647]: debug 2024-02-10T08:13:56.584+0000 7f0e4f31f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T08:13:58.055 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:56 smithi067 bash[17647]: debug 2024-02-10T08:13:56.844+0000 7f0e4f31f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T08:13:58.325 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:58 smithi067 bash[17647]: debug 2024-02-10T08:13:58.048+0000 7f0e4f31f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T08:13:58.325 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:58 smithi067 bash[17647]: debug 2024-02-10T08:13:58.220+0000 7f0e4f31f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:13:58.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:58 smithi067 bash[17647]: debug 2024-02-10T08:13:58.320+0000 7f0e4f31f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T08:13:58.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:58 smithi067 bash[17647]: debug 2024-02-10T08:13:58.436+0000 7f0e4f31f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T08:13:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 Processor -- start 2024-02-10T08:13:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- start start 2024-02-10T08:13:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e80fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e80fe270 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ed74c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e80fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:58.737 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ed74c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e80fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49970/0 (socket says 172.21.15.67:49970) 2024-02-10T08:13:58.737 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ed74c700 1 -- 172.21.15.67:0/1804710976 learned_addr learned my addr 172.21.15.67:0/1804710976 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:58.737 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ed74c700 1 -- 172.21.15.67:0/1804710976 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80fe3b0 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ed74c700 1 --2- 172.21.15.67:0/1804710976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e80fdcf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ff0e4009720 tx=0x7ff0e40090f0).ready entity=mon.0 client_cookie=59b04e3da501ba85 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0dbfff700 1 -- 172.21.15.67:0/1804710976 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e400b520 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0dbfff700 1 -- 172.21.15.67:0/1804710976 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff0e400baf0 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0dbfff700 1 -- 172.21.15.67:0/1804710976 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e40119f0 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/1804710976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 msgr2=0x7ff0e80fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 --2- 172.21.15.67:0/1804710976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e80fdcf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7ff0e4009720 tx=0x7ff0e40090f0).stop 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/1804710976 shutdown_connections 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 --2- 172.21.15.67:0/1804710976 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e80fdcf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/1804710976 >> 172.21.15.67:0/1804710976 conn(0x7ff0e80710b0 msgr2=0x7ff0e80714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/1804710976 shutdown_connections 2024-02-10T08:13:58.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.732+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/1804710976 wait complete. 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ef9b0700 1 Processor -- start 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ef9b0700 1 -- start start 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ef9b0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e810c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ef9b0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e810c940 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ed74c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e810c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ed74c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e810c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:49980/0 (socket says 172.21.15.67:49980) 2024-02-10T08:13:58.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ed74c700 1 -- 172.21.15.67:0/2004331662 learned_addr learned my addr 172.21.15.67:0/2004331662 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ed74c700 1 -- 172.21.15.67:0/2004331662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4009310 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ed74c700 1 --2- 172.21.15.67:0/2004331662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e810c450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7ff0e400bec0 tx=0x7ff0e40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e401a640 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff0e401ac10 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0e4022af0 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/2004331662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e810cb40 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0ef9b0700 1 -- 172.21.15.67:0/2004331662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e810cfd0 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7ff0e4023450 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.736+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff0e4019760 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.740+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c8004f20 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.740+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (secure 0 0 0) 0x7ff0e4042e50 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.882 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.876+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff0c80051d0 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.876+0000 7ff0da7fc700 1 -- 172.21.15.67:0/2004331662 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7ff0e4031030 con 0x7ff0e80fd8d0 2024-02-10T08:13:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:13:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "fsid": "2dd26d0c-c7ec-11ee-95b8-87774f69a715", 2024-02-10T08:13:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "health": { 2024-02-10T08:13:58.890 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "checks": {}, 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mutes": [] 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum": [ 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 0 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "a" 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "monmap": { 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-10T08:13:58.891 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osdmap": { 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "pgmap": { 2024-02-10T08:13:58.892 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "fsmap": { 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.893 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "available": false, 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "modules": [ 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "iostat", 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "nfs", 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "restful" 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "services": {} 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "servicemap": { 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "modified": "2024-02-10T08:13:45.490008+0000", 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "services": {} 2024-02-10T08:13:58.894 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 msgr2=0x7ff0e810c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 --2- 172.21.15.67:0/2004331662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e810c450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7ff0e400bec0 tx=0x7ff0e40090f0).stop 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 shutdown_connections 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 --2- 172.21.15.67:0/2004331662 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff0e80fd8d0 0x7ff0e810c450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 >> 172.21.15.67:0/2004331662 conn(0x7ff0e80710b0 msgr2=0x7ff0e80fc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 shutdown_connections 2024-02-10T08:13:58.895 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:13:58.880+0000 7ff0d3fff700 1 -- 172.21.15.67:0/2004331662 wait complete. 2024-02-10T08:13:58.933 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:58 smithi067 bash[17647]: debug 2024-02-10T08:13:58.664+0000 7f0e4f31f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T08:13:59.200 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:59 smithi067 bash[17647]: debug 2024-02-10T08:13:59.064+0000 7f0e4f31f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T08:13:59.201 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:13:58 smithi067 bash[17385]: audit 2024-02-10T08:13:58.882201+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.15.67:0/2004331662' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T08:13:59.537 INFO:teuthology.orchestra.run.smithi067.stderr:mgr not available, waiting (2/15)... 2024-02-10T08:13:59.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:13:59 smithi067 bash[17647]: debug 2024-02-10T08:13:59.196+0000 7f0e4f31f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T08:14:00.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:00 smithi067 bash[17647]: debug 2024-02-10T08:14:00.112+0000 7f0e4f31f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T08:14:00.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:00 smithi067 bash[17647]: debug 2024-02-10T08:14:00.220+0000 7f0e4f31f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T08:14:00.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:00 smithi067 bash[17647]: debug 2024-02-10T08:14:00.344+0000 7f0e4f31f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T08:14:00.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:00 smithi067 bash[17647]: debug 2024-02-10T08:14:00.572+0000 7f0e4f31f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T08:14:01.221 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:00 smithi067 bash[17647]: debug 2024-02-10T08:14:00.868+0000 7f0e4f31f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T08:14:01.221 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17647]: debug 2024-02-10T08:14:01.036+0000 7f0e4f31f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T08:14:01.551 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17647]: debug 2024-02-10T08:14:01.216+0000 7f0e4f31f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:14:01.551 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17647]: debug 2024-02-10T08:14:01.440+0000 7f0e4f31f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17647]: debug 2024-02-10T08:14:01.548+0000 7f0e4f31f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: cluster 2024-02-10T08:14:01.572197+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: cluster 2024-02-10T08:14:01.576732+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00464625s) 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: audit 2024-02-10T08:14:01.577857+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: audit 2024-02-10T08:14:01.578031+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: audit 2024-02-10T08:14:01.578192+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: audit 2024-02-10T08:14:01.578355+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:14:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: audit 2024-02-10T08:14:01.579090+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T08:14:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: cluster 2024-02-10T08:14:01.606103+0000 mon.a (mon.0) 16 : cluster [INF] Manager daemon y is now available 2024-02-10T08:14:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:01 smithi067 bash[17385]: audit 2024-02-10T08:14:01.614974+0000 mon.a (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 Processor -- start 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- start start 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d00faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00faf80 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0ce59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d00faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0ce59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d00faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50020/0 (socket says 172.21.15.67:50020) 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0ce59c700 1 -- 172.21.15.67:0/2137748801 learned_addr learned my addr 172.21.15.67:0/2137748801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0ce59c700 1 -- 172.21.15.67:0/2137748801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00fb0c0 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0ce59c700 1 --2- 172.21.15.67:0/2137748801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d00faa00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fe0c0009740 tx=0x7fe0c0009060).ready entity=mon.0 client_cookie=617ddbb015ce3d1f server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:02.587 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0cd59a700 1 -- 172.21.15.67:0/2137748801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0c000b520 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0cd59a700 1 -- 172.21.15.67:0/2137748801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fe0c000baf0 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0cd59a700 1 -- 172.21.15.67:0/2137748801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0c0011a10 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/2137748801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 msgr2=0x7fe0d00faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 --2- 172.21.15.67:0/2137748801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d00faa00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fe0c0009740 tx=0x7fe0c0009060).stop 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/2137748801 shutdown_connections 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 --2- 172.21.15.67:0/2137748801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d00faa00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/2137748801 >> 172.21.15.67:0/2137748801 conn(0x7fe0d00f5ce0 msgr2=0x7fe0d00f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/2137748801 shutdown_connections 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/2137748801 wait complete. 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 Processor -- start 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- start start 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d010c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.580+0000 7fe0d4d4a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d010c930 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0ce59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d010c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0ce59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d010c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50026/0 (socket says 172.21.15.67:50026) 2024-02-10T08:14:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0ce59c700 1 -- 172.21.15.67:0/1140755712 learned_addr learned my addr 172.21.15.67:0/1140755712 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0ce59c700 1 -- 172.21.15.67:0/1140755712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0009300 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.589 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0ce59c700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d010c440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fe0c0009220 tx=0x7fe0c0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0c000be30 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d010cb30 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fe0c001aa80 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0c00227a0 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d010cfc0 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.590 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 3) v1 ==== 38500+0+0 (secure 0 0 0) 0x7fe0c001abf0 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.591 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0bf7fe700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe0b8032440 0x7fe0b8034900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:02.591 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe0c0044700 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.591 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d0046d80 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.591 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0cdd9b700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe0b8032440 0x7fe0b8034900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:02.591 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.584+0000 7fe0cdd9b700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe0b8032440 0x7fe0b8034900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fe0c4009780 tx=0x7fe0c4006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:02.596 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:02.592+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe0c001e080 con 0x7fe0d00fa5e0 2024-02-10T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:02 smithi067 bash[17385]: audit 2024-02-10T08:14:01.628800+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:02 smithi067 bash[17385]: audit 2024-02-10T08:14:01.632193+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:02 smithi067 bash[17385]: audit 2024-02-10T08:14:01.635499+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' 2024-02-10T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:02 smithi067 bash[17385]: audit 2024-02-10T08:14:01.638747+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.67:0/2947371687' entity='mgr.y' 2024-02-10T08:14:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:02 smithi067 bash[17385]: cluster 2024-02-10T08:14:02.578379+0000 mon.a (mon.0) 22 : cluster [DBG] mgrmap e3: y(active, since 1.00628s) 2024-02-10T08:14:03.056 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.052+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe0d00fc3e0 con 0x7fe0d00fa5e0 2024-02-10T08:14:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.052+0000 7fe0bf7fe700 1 -- 172.21.15.67:0/1140755712 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (secure 0 0 0) 0x7fe0d00fc3e0 con 0x7fe0d00fa5e0 2024-02-10T08:14:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.059 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "fsid": "2dd26d0c-c7ec-11ee-95b8-87774f69a715", 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "health": { 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "checks": {}, 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mutes": [] 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum": [ 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 0 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-10T08:14:03.060 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "a" 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "quorum_age": 11, 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "monmap": { 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osdmap": { 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-10T08:14:03.061 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "pgmap": { 2024-02-10T08:14:03.066 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "fsmap": { 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-10T08:14:03.067 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "available": true, 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "modules": [ 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "iostat", 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "nfs", 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "restful" 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ], 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "services": {} 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "servicemap": { 2024-02-10T08:14:03.068 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "modified": "2024-02-10T08:13:45.490008+0000", 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "services": {} 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: }, 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe0b8032440 msgr2=0x7fe0b8034900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe0b8032440 0x7fe0b8034900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fe0c4009780 tx=0x7fe0c4006cc0).stop 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 msgr2=0x7fe0d010c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d010c440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fe0c0009220 tx=0x7fe0c0009060).stop 2024-02-10T08:14:03.069 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 shutdown_connections 2024-02-10T08:14:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe0b8032440 0x7fe0b8034900 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 --2- 172.21.15.67:0/1140755712 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe0d00fa5e0 0x7fe0d010c440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 >> 172.21.15.67:0/1140755712 conn(0x7fe0d00f5ce0 msgr2=0x7fe0d006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 shutdown_connections 2024-02-10T08:14:03.070 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:03.056+0000 7fe0d4d4a700 1 -- 172.21.15.67:0/1140755712 wait complete. 2024-02-10T08:14:03.702 INFO:teuthology.orchestra.run.smithi067.stderr:mgr is available 2024-02-10T08:14:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:03 smithi067 bash[17385]: audit 2024-02-10T08:14:03.056809+0000 mon.a (mon.0) 23 : audit [DBG] from='client.? 172.21.15.67:0/1140755712' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T08:14:04.749 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd7efa700 1 Processor -- start 2024-02-10T08:14:04.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd7efa700 1 -- start start 2024-02-10T08:14:04.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd7efa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd00fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:04.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd7efa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00fe270 con 0x7fccd00fd8d0 2024-02-10T08:14:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd5c96700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd00fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd5c96700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd00fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55736/0 (socket says 172.21.15.67:55736) 2024-02-10T08:14:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd5c96700 1 -- 172.21.15.67:0/410615410 learned_addr learned my addr 172.21.15.67:0/410615410 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:04.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.744+0000 7fccd5c96700 1 -- 172.21.15.67:0/410615410 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00fe3b0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd5c96700 1 --2- 172.21.15.67:0/410615410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd00fdcf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fcccc009730 tx=0x7fcccc0090f0).ready entity=mon.0 client_cookie=6df803b3339763c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd4c94700 1 -- 172.21.15.67:0/410615410 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcccc00b580 con 0x7fccd00fd8d0 2024-02-10T08:14:04.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd4c94700 1 -- 172.21.15.67:0/410615410 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fcccc00bb50 con 0x7fccd00fd8d0 2024-02-10T08:14:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd4c94700 1 -- 172.21.15.67:0/410615410 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcccc011a70 con 0x7fccd00fd8d0 2024-02-10T08:14:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- 172.21.15.67:0/410615410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 msgr2=0x7fccd00fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 --2- 172.21.15.67:0/410615410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd00fdcf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fcccc009730 tx=0x7fcccc0090f0).stop 2024-02-10T08:14:04.756 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- 172.21.15.67:0/410615410 shutdown_connections 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 --2- 172.21.15.67:0/410615410 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd00fdcf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- 172.21.15.67:0/410615410 >> 172.21.15.67:0/410615410 conn(0x7fccd0070e40 msgr2=0x7fccd0071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- 172.21.15.67:0/410615410 shutdown_connections 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- 172.21.15.67:0/410615410 wait complete. 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 Processor -- start 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- start start 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd010c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd7efa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd010c9e0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd5c96700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd010c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd5c96700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd010c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55752/0 (socket says 172.21.15.67:55752) 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd5c96700 1 -- 172.21.15.67:0/1359478801 learned_addr learned my addr 172.21.15.67:0/1359478801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:04.757 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.748+0000 7fccd5c96700 1 -- 172.21.15.67:0/1359478801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc009370 con 0x7fccd00fd8d0 2024-02-10T08:14:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccd5c96700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd010c4f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fcccc000c00 tx=0x7fcccc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcccc01a7b0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fcccc01ad80 con 0x7fccd00fd8d0 2024-02-10T08:14:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd010cbe0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcccc023bd0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.758 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd010d070 con 0x7fccd00fd8d0 2024-02-10T08:14:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 38606+0+0 (secure 0 0 0) 0x7fcccc02c3f0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccc6ffd700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fccbc0325e0 0x7fccbc034aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fcccc045080 con 0x7fccd00fd8d0 2024-02-10T08:14:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.752+0000 7fccd5495700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fccbc0325e0 0x7fccbc034aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.756+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd005a3b0 con 0x7fccd00fd8d0 2024-02-10T08:14:04.759 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.756+0000 7fccd5495700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fccbc0325e0 0x7fccbc034aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fccc0009780 tx=0x7fccc0006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:04.765 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:04.760+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fcccc01e020 con 0x7fccd00fd8d0 2024-02-10T08:14:04.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:04 smithi067 bash[17385]: cluster 2024-02-10T08:14:03.633570+0000 mon.a (mon.0) 24 : cluster [DBG] mgrmap e4: y(active, since 2s) 2024-02-10T08:14:05.092 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.088+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fccd0046d80 con 0x7fccd00fd8d0 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.088+0000 7fccc6ffd700 1 -- 172.21.15.67:0/1359478801 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+307 (secure 0 0 0) 0x7fcccc02b8e0 con 0x7fccd00fd8d0 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: [global] 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: fsid = 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: [mgr] 2024-02-10T08:14:05.094 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-10T08:14:05.095 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.095 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: [osd] 2024-02-10T08:14:05.095 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-10T08:14:05.095 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fccbc0325e0 msgr2=0x7fccbc034aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fccbc0325e0 0x7fccbc034aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fccc0009780 tx=0x7fccc0006cc0).stop 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 msgr2=0x7fccd010c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd010c4f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fcccc000c00 tx=0x7fcccc0090f0).stop 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 shutdown_connections 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fccbc0325e0 0x7fccbc034aa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 --2- 172.21.15.67:0/1359478801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fccd00fd8d0 0x7fccd010c4f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 >> 172.21.15.67:0/1359478801 conn(0x7fccd0070e40 msgr2=0x7fccd00fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 shutdown_connections 2024-02-10T08:14:05.097 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:05.092+0000 7fccd7efa700 1 -- 172.21.15.67:0/1359478801 wait complete. 2024-02-10T08:14:05.855 INFO:teuthology.orchestra.run.smithi067.stderr:Enabling cephadm module... 2024-02-10T08:14:05.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:05 smithi067 bash[17385]: audit 2024-02-10T08:14:05.092301+0000 mon.a (mon.0) 25 : audit [INF] from='client.? 172.21.15.67:0/1359478801' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T08:14:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50ce54c700 1 Processor -- start 2024-02-10T08:14:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50ce54c700 1 -- start start 2024-02-10T08:14:06.838 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50ce54c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c80faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50ce54c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80faf80 con 0x7f50c80fa5e0 2024-02-10T08:14:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c80faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:06.839 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c80faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55762/0 (socket says 172.21.15.67:55762) 2024-02-10T08:14:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50c7fff700 1 -- 172.21.15.67:0/2403901035 learned_addr learned my addr 172.21.15.67:0/2403901035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.832+0000 7f50c7fff700 1 -- 172.21.15.67:0/2403901035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80fb0c0 con 0x7f50c80fa5e0 2024-02-10T08:14:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50c7fff700 1 --2- 172.21.15.67:0/2403901035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c80faa00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f50b8009730 tx=0x7f50b80090f0).ready entity=mon.0 client_cookie=e56dab3670541eba server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50c6ffd700 1 -- 172.21.15.67:0/2403901035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50b800b580 con 0x7f50c80fa5e0 2024-02-10T08:14:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50c6ffd700 1 -- 172.21.15.67:0/2403901035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f50b800bb50 con 0x7f50c80fa5e0 2024-02-10T08:14:06.840 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50c6ffd700 1 -- 172.21.15.67:0/2403901035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50b8011a70 con 0x7f50c80fa5e0 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 -- 172.21.15.67:0/2403901035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 msgr2=0x7f50c80faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 --2- 172.21.15.67:0/2403901035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c80faa00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f50b8009730 tx=0x7f50b80090f0).stop 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 -- 172.21.15.67:0/2403901035 shutdown_connections 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 --2- 172.21.15.67:0/2403901035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c80faa00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 -- 172.21.15.67:0/2403901035 >> 172.21.15.67:0/2403901035 conn(0x7f50c80f5d00 msgr2=0x7f50c80f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 -- 172.21.15.67:0/2403901035 shutdown_connections 2024-02-10T08:14:06.841 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 -- 172.21.15.67:0/2403901035 wait complete. 2024-02-10T08:14:06.842 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 Processor -- start 2024-02-10T08:14:06.842 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.836+0000 7f50ce54c700 1 -- start start 2024-02-10T08:14:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50ce54c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c810d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50ce54c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c806fc60 con 0x7f50c80fa5e0 2024-02-10T08:14:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c810d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c810d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55772/0 (socket says 172.21.15.67:55772) 2024-02-10T08:14:06.843 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c7fff700 1 -- 172.21.15.67:0/3885512786 learned_addr learned my addr 172.21.15.67:0/3885512786 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c7fff700 1 -- 172.21.15.67:0/3885512786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8009370 con 0x7f50c80fa5e0 2024-02-10T08:14:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c7fff700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c810d020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f50b8009f00 tx=0x7f50b80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50b801a640 con 0x7f50c80fa5e0 2024-02-10T08:14:06.844 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f50b801ac10 con 0x7f50c80fa5e0 2024-02-10T08:14:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c806fe60 con 0x7f50c80fa5e0 2024-02-10T08:14:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50b8023b10 con 0x7f50c80fa5e0 2024-02-10T08:14:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c810d6d0 con 0x7f50c80fa5e0 2024-02-10T08:14:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 4) v1 ==== 38606+0+0 (secure 0 0 0) 0x7f50b801a7a0 con 0x7f50c80fa5e0 2024-02-10T08:14:06.845 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c57fa700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7f50b00325e0 0x7f50b0034aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f50b8044c60 con 0x7f50c80fa5e0 2024-02-10T08:14:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.840+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c805a3b0 con 0x7f50c80fa5e0 2024-02-10T08:14:06.846 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.844+0000 7f50c77fe700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7f50b00325e0 0x7f50b0034aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:06.847 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.844+0000 7f50c77fe700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7f50b00325e0 0x7f50b0034aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f50bc009780 tx=0x7f50bc006cc0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:06.853 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:06.848+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f50b801e020 con 0x7f50c80fa5e0 2024-02-10T08:14:07.266 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.260+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f50c80fc520 con 0x7f50c80fa5e0 2024-02-10T08:14:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.964+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 5) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f50b802c3f0 con 0x7f50c80fa5e0 2024-02-10T08:14:07.968 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.964+0000 7f50c57fa700 1 -- 172.21.15.67:0/3885512786 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f50b800f510 con 0x7f50c80fa5e0 2024-02-10T08:14:07.972 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7f50b00325e0 msgr2=0x7f50b0034aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:07.972 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7f50b00325e0 0x7f50b0034aa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f50bc009780 tx=0x7f50bc006cc0).stop 2024-02-10T08:14:07.972 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 msgr2=0x7f50c810d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c810d020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f50b8009f00 tx=0x7f50b80090f0).stop 2024-02-10T08:14:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 shutdown_connections 2024-02-10T08:14:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7f50b00325e0 0x7f50b0034aa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 --2- 172.21.15.67:0/3885512786 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f50c80fa5e0 0x7f50c810d020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 >> 172.21.15.67:0/3885512786 conn(0x7f50c80f5d00 msgr2=0x7f50c80f7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:07.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 shutdown_connections 2024-02-10T08:14:07.974 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:07.968+0000 7f50ce54c700 1 -- 172.21.15.67:0/3885512786 wait complete. 2024-02-10T08:14:08.244 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:07 smithi067 bash[17385]: audit 2024-02-10T08:14:07.266512+0000 mon.a (mon.0) 26 : audit [INF] from='client.? 172.21.15.67:0/3885512786' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-10T08:14:08.244 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17647]: ignoring --setuser ceph since I am not root 2024-02-10T08:14:08.244 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17647]: ignoring --setgroup ceph since I am not root 2024-02-10T08:14:08.244 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17647]: debug 2024-02-10T08:14:08.172+0000 7f6611045680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T08:14:08.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17647]: debug 2024-02-10T08:14:08.280+0000 7f6611045680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T08:14:08.966 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17647]: debug 2024-02-10T08:14:08.756+0000 7f6611045680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T08:14:09.369 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17385]: audit 2024-02-10T08:14:07.968345+0000 mon.a (mon.0) 27 : audit [INF] from='client.? 172.21.15.67:0/3885512786' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-10T08:14:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:08 smithi067 bash[17385]: cluster 2024-02-10T08:14:07.968402+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e5: y(active, since 6s) 2024-02-10T08:14:09.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.736+0000 7fe410924700 1 Processor -- start 2024-02-10T08:14:09.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.736+0000 7fe410924700 1 -- start start 2024-02-10T08:14:09.742 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.736+0000 7fe410924700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c0f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:09.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.736+0000 7fe410924700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0fa350 con 0x7fe40c0f99b0 2024-02-10T08:14:09.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c0f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:09.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c0f9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55798/0 (socket says 172.21.15.67:55798) 2024-02-10T08:14:09.743 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40a59c700 1 -- 172.21.15.67:0/17085978 learned_addr learned my addr 172.21.15.67:0/17085978 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:09.744 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40a59c700 1 -- 172.21.15.67:0/17085978 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c0fa490 con 0x7fe40c0f99b0 2024-02-10T08:14:09.744 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40a59c700 1 --2- 172.21.15.67:0/17085978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c0f9dd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe3fc009730 tx=0x7fe3fc0090f0).ready entity=mon.0 client_cookie=d030e0a5311bd89 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:09.744 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40959a700 1 -- 172.21.15.67:0/17085978 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3fc00b580 con 0x7fe40c0f99b0 2024-02-10T08:14:09.744 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40959a700 1 -- 172.21.15.67:0/17085978 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fe3fc00bb50 con 0x7fe40c0f99b0 2024-02-10T08:14:09.744 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe40959a700 1 -- 172.21.15.67:0/17085978 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3fc011a70 con 0x7fe40c0f99b0 2024-02-10T08:14:09.745 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 -- 172.21.15.67:0/17085978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 msgr2=0x7fe40c0f9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:09.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 --2- 172.21.15.67:0/17085978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c0f9dd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe3fc009730 tx=0x7fe3fc0090f0).stop 2024-02-10T08:14:09.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 -- 172.21.15.67:0/17085978 shutdown_connections 2024-02-10T08:14:09.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 --2- 172.21.15.67:0/17085978 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c0f9dd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:09.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 -- 172.21.15.67:0/17085978 >> 172.21.15.67:0/17085978 conn(0x7fe40c0f5460 msgr2=0x7fe40c0f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:09.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 -- 172.21.15.67:0/17085978 shutdown_connections 2024-02-10T08:14:09.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.740+0000 7fe410924700 1 -- 172.21.15.67:0/17085978 wait complete. 2024-02-10T08:14:09.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe410924700 1 Processor -- start 2024-02-10T08:14:09.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe410924700 1 -- start start 2024-02-10T08:14:09.748 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe410924700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c10a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:09.748 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe410924700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c10a780 con 0x7fe40c0f99b0 2024-02-10T08:14:09.748 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe40a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c10a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:09.748 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe40a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c10a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55802/0 (socket says 172.21.15.67:55802) 2024-02-10T08:14:09.748 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe40a59c700 1 -- 172.21.15.67:0/2386411297 learned_addr learned my addr 172.21.15.67:0/2386411297 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:09.749 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe40a59c700 1 -- 172.21.15.67:0/2386411297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc009370 con 0x7fe40c0f99b0 2024-02-10T08:14:09.749 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe40a59c700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c10a290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe3fc000c00 tx=0x7fe3fc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3fc01a7b0 con 0x7fe40c0f99b0 2024-02-10T08:14:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c10a980 con 0x7fe40c0f99b0 2024-02-10T08:14:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fe3fc01ad80 con 0x7fe40c0f99b0 2024-02-10T08:14:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.744+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3fc023bd0 con 0x7fe40c0f99b0 2024-02-10T08:14:09.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c10ae10 con 0x7fe40c0f99b0 2024-02-10T08:14:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c046d80 con 0x7fe40c0f99b0 2024-02-10T08:14:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fe3fc0223c0 con 0x7fe40c0f99b0 2024-02-10T08:14:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe4037fe700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 0x7fe3f8034890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:09.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7fe3fc04e6c0 con 0x7fe40c0f99b0 2024-02-10T08:14:09.753 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe409d9b700 1 -- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 msgr2=0x7fe3f8034890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:09.753 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.748+0000 7fe409d9b700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 0x7fe3f8034890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:14:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.756+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe3fc04b060 con 0x7fe40c0f99b0 2024-02-10T08:14:09.818 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:09 smithi067 bash[17647]: debug 2024-02-10T08:14:09.704+0000 7f6611045680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T08:14:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.952+0000 7fe409d9b700 1 -- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 msgr2=0x7fe3f8034890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:09.955 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:09.952+0000 7fe409d9b700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 0x7fe3f8034890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-10T08:14:10.074 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:09 smithi067 bash[17647]: debug 2024-02-10T08:14:09.812+0000 7f6611045680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T08:14:10.171 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.168+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe40c103f90 con 0x7fe40c0f99b0 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.168+0000 7fe4037fe700 1 -- 172.21.15.67:0/2386411297 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe3fc04b020 con 0x7fe40c0f99b0 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "available": true, 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-10T08:14:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:14:10.177 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 msgr2=0x7fe3f8034890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:14:10.177 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 0x7fe3f8034890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:10.177 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 msgr2=0x7fe40c10a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c10a290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe3fc000c00 tx=0x7fe3fc0090f0).stop 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 shutdown_connections 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7fe3f80323d0 0x7fe3f8034890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 --2- 172.21.15.67:0/2386411297 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe40c0f99b0 0x7fe40c10a290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 >> 172.21.15.67:0/2386411297 conn(0x7fe40c0f5460 msgr2=0x7fe40c0f6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 shutdown_connections 2024-02-10T08:14:10.178 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:10.172+0000 7fe410924700 1 -- 172.21.15.67:0/2386411297 wait complete. 2024-02-10T08:14:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:10 smithi067 bash[17385]: audit 2024-02-10T08:14:10.172105+0000 mon.a (mon.0) 29 : audit [DBG] from='client.? 172.21.15.67:0/2386411297' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T08:14:10.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:10 smithi067 bash[17647]: debug 2024-02-10T08:14:10.068+0000 7f6611045680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T08:14:10.892 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for the mgr to restart... 2024-02-10T08:14:10.892 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for mgr epoch 5... 2024-02-10T08:14:11.533 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:11 smithi067 bash[17647]: debug 2024-02-10T08:14:11.260+0000 7f6611045680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T08:14:11.534 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:11 smithi067 bash[17647]: debug 2024-02-10T08:14:11.432+0000 7f6611045680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:14:11.731 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 Processor -- start 2024-02-10T08:14:11.732 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 -- start start 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff5180fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180fbfd0 con 0x7ff5180fb6c0 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff516ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff5180fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff516ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff5180fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55808/0 (socket says 172.21.15.67:55808) 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff516ffd700 1 -- 172.21.15.67:0/288578003 learned_addr learned my addr 172.21.15.67:0/288578003 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff516ffd700 1 -- 172.21.15.67:0/288578003 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5180fc110 con 0x7ff5180fb6c0 2024-02-10T08:14:11.733 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff516ffd700 1 --2- 172.21.15.67:0/288578003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff5180fbae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7ff508009730 tx=0x7ff5080090f0).ready entity=mon.0 client_cookie=7f6d11aca5946df6 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:11.734 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff515ffb700 1 -- 172.21.15.67:0/288578003 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff50800b580 con 0x7ff5180fb6c0 2024-02-10T08:14:11.734 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff515ffb700 1 -- 172.21.15.67:0/288578003 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff50800bb50 con 0x7ff5180fb6c0 2024-02-10T08:14:11.734 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff515ffb700 1 -- 172.21.15.67:0/288578003 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff508011a70 con 0x7ff5180fb6c0 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 -- 172.21.15.67:0/288578003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 msgr2=0x7ff5180fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 --2- 172.21.15.67:0/288578003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff5180fbae0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7ff508009730 tx=0x7ff5080090f0).stop 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 -- 172.21.15.67:0/288578003 shutdown_connections 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 --2- 172.21.15.67:0/288578003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff5180fbae0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 -- 172.21.15.67:0/288578003 >> 172.21.15.67:0/288578003 conn(0x7ff5180f6ec0 msgr2=0x7ff5180f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.728+0000 7ff51d9df700 1 -- 172.21.15.67:0/288578003 shutdown_connections 2024-02-10T08:14:11.735 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 -- 172.21.15.67:0/288578003 wait complete. 2024-02-10T08:14:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 Processor -- start 2024-02-10T08:14:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 -- start start 2024-02-10T08:14:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff51810c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51810c9d0 con 0x7ff5180fb6c0 2024-02-10T08:14:11.736 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff516ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff51810c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff516ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff51810c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55812/0 (socket says 172.21.15.67:55812) 2024-02-10T08:14:11.737 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff516ffd700 1 -- 172.21.15.67:0/2961725799 learned_addr learned my addr 172.21.15.67:0/2961725799 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff516ffd700 1 -- 172.21.15.67:0/2961725799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508009370 con 0x7ff5180fb6c0 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff516ffd700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff51810c4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff508006940 tx=0x7ff5080090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff50801a640 con 0x7ff5180fb6c0 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff50801ac10 con 0x7ff5180fb6c0 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51810cbd0 con 0x7ff5180fb6c0 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff508023b90 con 0x7ff5180fb6c0 2024-02-10T08:14:11.738 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51810d060 con 0x7ff5180fb6c0 2024-02-10T08:14:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.732+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 5) v1 ==== 38617+0+0 (secure 0 0 0) 0x7ff508023cf0 con 0x7ff5180fb6c0 2024-02-10T08:14:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.736+0000 7ff4fffff700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.736+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 --> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff51805a3b0 con 0x7ff500032610 2024-02-10T08:14:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.736+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7ff508044df0 con 0x7ff5180fb6c0 2024-02-10T08:14:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.736+0000 7ff5167fc700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 msgr2=0x7ff500034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:11.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.736+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:14:11.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:11 smithi067 bash[17647]: debug 2024-02-10T08:14:11.536+0000 7f6611045680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T08:14:11.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:11 smithi067 bash[17647]: debug 2024-02-10T08:14:11.652+0000 7f6611045680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T08:14:11.943 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.940+0000 7ff5167fc700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 msgr2=0x7ff500034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:11.943 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:11.940+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-10T08:14:12.321 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:11 smithi067 bash[17647]: debug 2024-02-10T08:14:11.904+0000 7f6611045680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T08:14:12.347 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:12.344+0000 7ff5167fc700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 msgr2=0x7ff500034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:12.347 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:12.344+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-10T08:14:12.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:12 smithi067 bash[17647]: debug 2024-02-10T08:14:12.316+0000 7f6611045680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T08:14:12.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:12 smithi067 bash[17647]: debug 2024-02-10T08:14:12.456+0000 7f6611045680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T08:14:13.151 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:13.148+0000 7ff5167fc700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 msgr2=0x7ff500034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:13.151 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:13.148+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-10T08:14:13.841 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:13 smithi067 bash[17647]: debug 2024-02-10T08:14:13.380+0000 7f6611045680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T08:14:14.004 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:13 smithi067 bash[17647]: debug 2024-02-10T08:14:13.488+0000 7f6611045680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T08:14:14.004 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:13 smithi067 bash[17647]: debug 2024-02-10T08:14:13.604+0000 7f6611045680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T08:14:14.489 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:13 smithi067 bash[17647]: debug 2024-02-10T08:14:13.836+0000 7f6611045680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T08:14:14.489 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17647]: debug 2024-02-10T08:14:14.136+0000 7f6611045680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T08:14:14.489 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17647]: debug 2024-02-10T08:14:14.304+0000 7f6611045680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T08:14:14.756 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:14.752+0000 7ff5167fc700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 msgr2=0x7ff500034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2812373217 2024-02-10T08:14:14.765 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:14.752+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-02-10T08:14:14.820 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17647]: debug 2024-02-10T08:14:14.484+0000 7f6611045680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:14:14.821 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17647]: debug 2024-02-10T08:14:14.704+0000 7f6611045680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T08:14:14.847 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:14.840+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 6) v1 ==== 38441+0+0 (secure 0 0 0) 0x7ff508029d00 con 0x7ff5180fb6c0 2024-02-10T08:14:14.847 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:14.840+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 msgr2=0x7ff500034ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:14:14.847 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:14.840+0000 7ff4fffff700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17647]: debug 2024-02-10T08:14:14.816+0000 7f6611045680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: cluster 2024-02-10T08:14:14.841219+0000 mon.a (mon.0) 30 : cluster [INF] Active manager daemon y restarted 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: cluster 2024-02-10T08:14:14.841637+0000 mon.a (mon.0) 31 : cluster [INF] Activating manager daemon y 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: cluster 2024-02-10T08:14:14.848488+0000 mon.a (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: cluster 2024-02-10T08:14:14.848770+0000 mon.a (mon.0) 33 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00729014s) 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: audit 2024-02-10T08:14:14.849897+0000 mon.a (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:14:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: audit 2024-02-10T08:14:14.850950+0000 mon.a (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T08:14:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: audit 2024-02-10T08:14:14.851834+0000 mon.a (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T08:14:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: audit 2024-02-10T08:14:14.852055+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T08:14:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: audit 2024-02-10T08:14:14.852269+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T08:14:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:14 smithi067 bash[17385]: cluster 2024-02-10T08:14:14.874420+0000 mon.a (mon.0) 39 : cluster [INF] Manager daemon y is now available 2024-02-10T08:14:15.849 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.844+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 7) v1 ==== 38511+0+0 (secure 0 0 0) 0x7ff50802cc90 con 0x7ff5180fb6c0 2024-02-10T08:14:15.849 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.844+0000 7ff4fffff700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff500035f30 0x7ff500038320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:15.849 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.844+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff51805a3b0 con 0x7ff500035f30 2024-02-10T08:14:15.850 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.844+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff500035f30 0x7ff500038320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:15.851 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.848+0000 7ff5167fc700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff500035f30 0x7ff500038320 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7ff50c0038a0 tx=0x7ff50c009350).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:15.852 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.848+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7ff51805a3b0 con 0x7ff500035f30 2024-02-10T08:14:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff518046d80 con 0x7ff500035f30 2024-02-10T08:14:15.865 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff4fffff700 1 -- 172.21.15.67:0/2961725799 <== mgr.14116 v2:172.21.15.67:6800/2933691832 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff518046d80 con 0x7ff500035f30 2024-02-10T08:14:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:14:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-10T08:14:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "initialized": true 2024-02-10T08:14:15.866 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff500035f30 msgr2=0x7ff500038320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff500035f30 0x7ff500038320 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7ff50c0038a0 tx=0x7ff50c009350).stop 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 msgr2=0x7ff51810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff51810c4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7ff508006940 tx=0x7ff5080090f0).stop 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 shutdown_connections 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff500035f30 0x7ff500038320 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff5180fb6c0 0x7ff51810c4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 --2- 172.21.15.67:0/2961725799 >> [v2:172.21.15.67:6800/2812373217,v1:172.21.15.67:6801/2812373217] conn(0x7ff500032610 0x7ff500034ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.860+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 >> 172.21.15.67:0/2961725799 conn(0x7ff5180f6ec0 msgr2=0x7ff5180f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.864+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 shutdown_connections 2024-02-10T08:14:15.869 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:15.864+0000 7ff51d9df700 1 -- 172.21.15.67:0/2961725799 wait complete. 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: cephadm 2024-02-10T08:14:14.890296+0000 mgr.y (mgr.14116) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:14.895804+0000 mon.a (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:15.026296+0000 mon.a (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:15.030366+0000 mon.a (mon.0) 42 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:15.034873+0000 mon.a (mon.0) 43 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:15.048518+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:15.065704+0000 mon.a (mon.0) 45 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: audit 2024-02-10T08:14:15.070661+0000 mon.a (mon.0) 46 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:14:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:15 smithi067 bash[17385]: cluster 2024-02-10T08:14:15.852015+0000 mon.a (mon.0) 47 : cluster [DBG] mgrmap e7: y(active, since 1.01054s) 2024-02-10T08:14:16.513 INFO:teuthology.orchestra.run.smithi067.stderr:mgr epoch 5 is available 2024-02-10T08:14:16.514 INFO:teuthology.orchestra.run.smithi067.stderr:Setting orchestrator backend to cephadm... 2024-02-10T08:14:17.186 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:16 smithi067 bash[17385]: audit 2024-02-10T08:14:15.852425+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T08:14:17.187 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:16 smithi067 bash[17385]: audit 2024-02-10T08:14:15.865527+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T08:14:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ec3dd700 1 Processor -- start 2024-02-10T08:14:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ec3dd700 1 -- start start 2024-02-10T08:14:17.421 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ec3dd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e40fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ec3dd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e40fbfd0 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ea179700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e40fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ea179700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e40fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57248/0 (socket says 172.21.15.67:57248) 2024-02-10T08:14:17.422 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ea179700 1 -- 172.21.15.67:0/3480815035 learned_addr learned my addr 172.21.15.67:0/3480815035 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ea179700 1 -- 172.21.15.67:0/3480815035 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40fc110 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1ea179700 1 --2- 172.21.15.67:0/3480815035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e40fbae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fd1e0009730 tx=0x7fd1e00090f0).ready entity=mon.0 client_cookie=5b5089f79be32df4 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1e9177700 1 -- 172.21.15.67:0/3480815035 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e000b580 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.423 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1e9177700 1 -- 172.21.15.67:0/3480815035 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fd1e000bb50 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.424 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.416+0000 7fd1e9177700 1 -- 172.21.15.67:0/3480815035 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0011a70 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.424 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/3480815035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 msgr2=0x7fd1e40fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:17.424 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 --2- 172.21.15.67:0/3480815035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e40fbae0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fd1e0009730 tx=0x7fd1e00090f0).stop 2024-02-10T08:14:17.424 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/3480815035 shutdown_connections 2024-02-10T08:14:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 --2- 172.21.15.67:0/3480815035 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e40fbae0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/3480815035 >> 172.21.15.67:0/3480815035 conn(0x7fd1e40f6ec0 msgr2=0x7fd1e40f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/3480815035 shutdown_connections 2024-02-10T08:14:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/3480815035 wait complete. 2024-02-10T08:14:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 Processor -- start 2024-02-10T08:14:17.425 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- start start 2024-02-10T08:14:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e410c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e410c9d0 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ea179700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e410c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ea179700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e410c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57262/0 (socket says 172.21.15.67:57262) 2024-02-10T08:14:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ea179700 1 -- 172.21.15.67:0/149403 learned_addr learned my addr 172.21.15.67:0/149403 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ea179700 1 -- 172.21.15.67:0/149403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0009370 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ea179700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e410c4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fd1e0009f00 tx=0x7fd1e00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e001a760 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fd1e001ad30 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e410cbd0 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.420+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0023bd0 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e410d060 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fd1e002c3f0 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1db7fe700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd1d00325e0 0x7fd1d0034aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fd1e0045360 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e405a3b0 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1e9978700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd1d00325e0 0x7fd1d0034aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:17.429 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.424+0000 7fd1e9978700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd1d00325e0 0x7fd1d0034aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fd1d4009780 tx=0x7fd1d4006cb0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:17.434 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.428+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd1e001e020 con 0x7fd1e40fb6c0 2024-02-10T08:14:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.780+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e40fd5a0 con 0x7fd1d00325e0 2024-02-10T08:14:17.794 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.792+0000 7fd1db7fe700 1 -- 172.21.15.67:0/149403 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd1e40fd5a0 con 0x7fd1d00325e0 2024-02-10T08:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd1d00325e0 msgr2=0x7fd1d0034aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd1d00325e0 0x7fd1d0034aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fd1d4009780 tx=0x7fd1d4006cb0).stop 2024-02-10T08:14:17.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 msgr2=0x7fd1e410c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e410c4e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fd1e0009f00 tx=0x7fd1e00090f0).stop 2024-02-10T08:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 shutdown_connections 2024-02-10T08:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd1d00325e0 0x7fd1d0034aa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 --2- 172.21.15.67:0/149403 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd1e40fb6c0 0x7fd1e410c4e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 >> 172.21.15.67:0/149403 conn(0x7fd1e40f6ec0 msgr2=0x7fd1e40f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 shutdown_connections 2024-02-10T08:14:17.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:17.796+0000 7fd1ec3dd700 1 -- 172.21.15.67:0/149403 wait complete. 2024-02-10T08:14:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:17 smithi067 bash[17385]: cluster 2024-02-10T08:14:16.904726+0000 mon.a (mon.0) 48 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-02-10T08:14:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:17 smithi067 bash[17385]: audit 2024-02-10T08:14:17.793829+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:17 smithi067 bash[17385]: audit 2024-02-10T08:14:17.824696+0000 mon.a (mon.0) 50 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:19.166 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:18 smithi067 bash[17385]: audit 2024-02-10T08:14:17.786817+0000 mgr.y (mgr.14116) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5c52e6700 1 Processor -- start 2024-02-10T08:14:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5c52e6700 1 -- start start 2024-02-10T08:14:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5c52e6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:19.398 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5c52e6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00fe270 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5beffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:19.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5beffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57264/0 (socket says 172.21.15.67:57264) 2024-02-10T08:14:19.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5beffd700 1 -- 172.21.15.67:0/2460732665 learned_addr learned my addr 172.21.15.67:0/2460732665 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:19.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5beffd700 1 -- 172.21.15.67:0/2460732665 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00fe3b0 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.392+0000 7fd5beffd700 1 --2- 172.21.15.67:0/2460732665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00fdcf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd5b0009730 tx=0x7fd5b00090f0).ready entity=mon.0 client_cookie=6273391e6b8a89e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:19.400 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5bdffb700 1 -- 172.21.15.67:0/2460732665 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b000b580 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5bdffb700 1 -- 172.21.15.67:0/2460732665 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fd5b000bb50 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5bdffb700 1 -- 172.21.15.67:0/2460732665 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b0011a70 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- 172.21.15.67:0/2460732665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 msgr2=0x7fd5c00fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 --2- 172.21.15.67:0/2460732665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00fdcf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd5b0009730 tx=0x7fd5b00090f0).stop 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- 172.21.15.67:0/2460732665 shutdown_connections 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 --2- 172.21.15.67:0/2460732665 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00fdcf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:19.401 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- 172.21.15.67:0/2460732665 >> 172.21.15.67:0/2460732665 conn(0x7fd5c0070e40 msgr2=0x7fd5c0071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- 172.21.15.67:0/2460732665 shutdown_connections 2024-02-10T08:14:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- 172.21.15.67:0/2460732665 wait complete. 2024-02-10T08:14:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 Processor -- start 2024-02-10T08:14:19.402 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- start start 2024-02-10T08:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00726a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5c52e6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0072b90 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5beffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5beffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57274/0 (socket says 172.21.15.67:57274) 2024-02-10T08:14:19.403 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5beffd700 1 -- 172.21.15.67:0/1373392745 learned_addr learned my addr 172.21.15.67:0/1373392745 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.396+0000 7fd5beffd700 1 -- 172.21.15.67:0/1373392745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0009370 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5beffd700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00726a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd5b0000c00 tx=0x7fd5b00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b001a7b0 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7fd5b001ad80 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c0072d90 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5b0023bd0 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c0073220 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.404 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fd5b0023410 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5b7fff700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd5ac032630 0x7fd5ac034af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fd5b0023e40 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5be7fc700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd5ac032630 0x7fd5ac034af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c005a3b0 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.406 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.400+0000 7fd5be7fc700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd5ac032630 0x7fd5ac034af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd5a8009780 tx=0x7fd5a8006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.408+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd5b0019df0 con 0x7fd5c00fd8d0 2024-02-10T08:14:19.740 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.736+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5c0073ad0 con 0x7fd5ac032630 2024-02-10T08:14:19.741 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.736+0000 7fd5b7fff700 1 -- 172.21.15.67:0/1373392745 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd5c0073ad0 con 0x7fd5ac032630 2024-02-10T08:14:19.741 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: value unchanged 2024-02-10T08:14:19.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd5ac032630 msgr2=0x7fd5ac034af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:19.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd5ac032630 0x7fd5ac034af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd5a8009780 tx=0x7fd5a8006cc0).stop 2024-02-10T08:14:19.746 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 msgr2=0x7fd5c00726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:19.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00726a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd5b0000c00 tx=0x7fd5b00090f0).stop 2024-02-10T08:14:19.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 shutdown_connections 2024-02-10T08:14:19.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fd5ac032630 0x7fd5ac034af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:19.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 --2- 172.21.15.67:0/1373392745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd5c00fd8d0 0x7fd5c00726a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:19.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 >> 172.21.15.67:0/1373392745 conn(0x7fd5c0070e40 msgr2=0x7fd5c00fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:19.747 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 shutdown_connections 2024-02-10T08:14:19.748 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:19.740+0000 7fd5c52e6700 1 -- 172.21.15.67:0/1373392745 wait complete. 2024-02-10T08:14:20.340 INFO:teuthology.orchestra.run.smithi067.stderr:Generating ssh key... 2024-02-10T08:14:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:20 smithi067 bash[17385]: audit 2024-02-10T08:14:19.740717+0000 mgr.y (mgr.14116) 5 : audit [DBG] from='client.14128 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:21.289 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fc51e4700 1 Processor -- start 2024-02-10T08:14:21.289 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fc51e4700 1 -- start start 2024-02-10T08:14:21.289 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fc51e4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc0070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:21.289 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fc51e4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00720c0 con 0x7f8fc0071ca0 2024-02-10T08:14:21.290 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc0070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:21.290 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc0070240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57278/0 (socket says 172.21.15.67:57278) 2024-02-10T08:14:21.290 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbed9d700 1 -- 172.21.15.67:0/1417600854 learned_addr learned my addr 172.21.15.67:0/1417600854 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:21.290 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbed9d700 1 -- 172.21.15.67:0/1417600854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0070730 con 0x7f8fc0071ca0 2024-02-10T08:14:21.290 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbed9d700 1 --2- 172.21.15.67:0/1417600854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc0070240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f8fb0009e50 tx=0x7f8fb0009840).ready entity=mon.0 client_cookie=28ce0aadbb068bce server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:21.291 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbdd9b700 1 -- 172.21.15.67:0/1417600854 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fb000e070 con 0x7f8fc0071ca0 2024-02-10T08:14:21.291 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbdd9b700 1 -- 172.21.15.67:0/1417600854 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8fb000be90 con 0x7f8fc0071ca0 2024-02-10T08:14:21.291 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.284+0000 7f8fbdd9b700 1 -- 172.21.15.67:0/1417600854 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fb0013d40 con 0x7f8fc0071ca0 2024-02-10T08:14:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1417600854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 msgr2=0x7f8fc0070240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:21.292 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 --2- 172.21.15.67:0/1417600854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc0070240 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f8fb0009e50 tx=0x7f8fb0009840).stop 2024-02-10T08:14:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1417600854 shutdown_connections 2024-02-10T08:14:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 --2- 172.21.15.67:0/1417600854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc0070240 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1417600854 >> 172.21.15.67:0/1417600854 conn(0x7f8fc00f6ec0 msgr2=0x7f8fc00f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1417600854 shutdown_connections 2024-02-10T08:14:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1417600854 wait complete. 2024-02-10T08:14:21.293 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 Processor -- start 2024-02-10T08:14:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- start start 2024-02-10T08:14:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc010c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fc51e4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc010ca30 con 0x7f8fc0071ca0 2024-02-10T08:14:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc010c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:21.294 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fbed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc010c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57290/0 (socket says 172.21.15.67:57290) 2024-02-10T08:14:21.295 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fbed9d700 1 -- 172.21.15.67:0/1592116470 learned_addr learned my addr 172.21.15.67:0/1592116470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.288+0000 7f8fbed9d700 1 -- 172.21.15.67:0/1592116470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0009ae0 con 0x7f8fc0071ca0 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fbed9d700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc010c540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8fb0009fb0 tx=0x7f8fb0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fb000e040 con 0x7f8fc0071ca0 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f8fb0012900 con 0x7f8fc0071ca0 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc010cc30 con 0x7f8fc0071ca0 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fb0007850 con 0x7f8fc0071ca0 2024-02-10T08:14:21.296 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc010d0c0 con 0x7f8fc0071ca0 2024-02-10T08:14:21.299 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f8fb0003040 con 0x7f8fc0071ca0 2024-02-10T08:14:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fa7fff700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f8fa80323c0 0x7f8fa8034880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f8fb0048630 con 0x7f8fc0071ca0 2024-02-10T08:14:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fbe59c700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f8fa80323c0 0x7f8fa8034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:21.300 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fbe59c700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f8fa80323c0 0x7f8fa8034880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8fb4009780 tx=0x7f8fb4006cb0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:21.301 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.292+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc005a3b0 con 0x7f8fc0071ca0 2024-02-10T08:14:21.305 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.300+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8fb0024b50 con 0x7f8fc0071ca0 2024-02-10T08:14:21.637 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.632+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fc010d920 con 0x7f8fa80323c0 2024-02-10T08:14:21.867 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fa7fff700 1 -- 172.21.15.67:0/1592116470 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8fc010d920 con 0x7f8fa80323c0 2024-02-10T08:14:21.871 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f8fa80323c0 msgr2=0x7f8fa8034880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:21.871 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f8fa80323c0 0x7f8fa8034880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f8fb4009780 tx=0x7f8fb4006cb0).stop 2024-02-10T08:14:21.871 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 msgr2=0x7f8fc010c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc010c540 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8fb0009fb0 tx=0x7f8fb0009840).stop 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 shutdown_connections 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f8fa80323c0 0x7f8fa8034880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 --2- 172.21.15.67:0/1592116470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8fc0071ca0 0x7f8fc010c540 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.864+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 >> 172.21.15.67:0/1592116470 conn(0x7f8fc00f6ec0 msgr2=0x7f8fc00f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.868+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 shutdown_connections 2024-02-10T08:14:21.872 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:21.868+0000 7f8fc51e4700 1 -- 172.21.15.67:0/1592116470 wait complete. 2024-02-10T08:14:22.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: Generating public/private rsa key pair. 2024-02-10T08:14:22.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: Your identification has been saved in /tmp/tmppu3tetxk/key. 2024-02-10T08:14:22.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: Your public key has been saved in /tmp/tmppu3tetxk/key.pub. 2024-02-10T08:14:22.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: The key fingerprint is: 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: SHA256:oOCWcEbzRA9RI0nuqvwroVeSUEN6zQR+66hFHQsrWHE ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: The key's randomart image is: 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: +---[RSA 3072]----+ 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: | .*+E+o | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: | +o@.+ . | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: |o.O.B o | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: |oB O = . | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: |o.*.* S | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: | =o+. | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: |. =o. | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: |o=. | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: |oooo. | 2024-02-10T08:14:22.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:21 smithi067 bash[17647]: +----[SHA256]-----+ 2024-02-10T08:14:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:22 smithi067 bash[17385]: audit 2024-02-10T08:14:21.638065+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:22 smithi067 bash[17385]: cephadm 2024-02-10T08:14:21.638506+0000 mgr.y (mgr.14116) 7 : cephadm [INF] Generating ssh key... 2024-02-10T08:14:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:22 smithi067 bash[17385]: audit 2024-02-10T08:14:21.862208+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:22 smithi067 bash[17385]: audit 2024-02-10T08:14:21.865308+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f83a3700 1 Processor -- start 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f83a3700 1 -- start start 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f83a3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f00fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f83a3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00fe270 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f613f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f00fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f613f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f00fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42790/0 (socket says 172.21.15.67:42790) 2024-02-10T08:14:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f613f700 1 -- 172.21.15.67:0/1473744475 learned_addr learned my addr 172.21.15.67:0/1473744475 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f613f700 1 -- 172.21.15.67:0/1473744475 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00fe3b0 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f613f700 1 --2- 172.21.15.67:0/1473744475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f00fdcf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff9e8009ea0 tx=0x7ff9e8009840).ready entity=mon.0 client_cookie=29f72c159d03321 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f513d700 1 -- 172.21.15.67:0/1473744475 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e800e070 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f513d700 1 -- 172.21.15.67:0/1473744475 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff9e800be70 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.443+0000 7ff9f513d700 1 -- 172.21.15.67:0/1473744475 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8013d60 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- 172.21.15.67:0/1473744475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 msgr2=0x7ff9f00fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:23.451 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 --2- 172.21.15.67:0/1473744475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f00fdcf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7ff9e8009ea0 tx=0x7ff9e8009840).stop 2024-02-10T08:14:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- 172.21.15.67:0/1473744475 shutdown_connections 2024-02-10T08:14:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 --2- 172.21.15.67:0/1473744475 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f00fdcf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- 172.21.15.67:0/1473744475 >> 172.21.15.67:0/1473744475 conn(0x7ff9f00710b0 msgr2=0x7ff9f00714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- 172.21.15.67:0/1473744475 shutdown_connections 2024-02-10T08:14:23.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- 172.21.15.67:0/1473744475 wait complete. 2024-02-10T08:14:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 Processor -- start 2024-02-10T08:14:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- start start 2024-02-10T08:14:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f010c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:23.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f83a3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f010c9b0 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f613f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f010c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f613f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f010c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42804/0 (socket says 172.21.15.67:42804) 2024-02-10T08:14:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f613f700 1 -- 172.21.15.67:0/2004378567 learned_addr learned my addr 172.21.15.67:0/2004378567 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f613f700 1 -- 172.21.15.67:0/2004378567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009ae0 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9f613f700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f010c4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ff9e8014f80 tx=0x7ff9e8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.447+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e800e040 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7ff9e8012900 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f010cbb0 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f010d040 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9e8007820 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7ff9e8003040 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc004f20 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9ef7fe700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff9d8032630 0x7ff9d8034af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:23.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7ff9e8044840 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9f593e700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff9d8032630 0x7ff9d8034af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:23.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.451+0000 7ff9f593e700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff9d8032630 0x7ff9d8034af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7ff9e4009780 tx=0x7ff9e4006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:23.463 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.459+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff9e8023300 con 0x7ff9f00fd8d0 2024-02-10T08:14:23.799 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.791+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9dc000bc0 con 0x7ff9d8032630 2024-02-10T08:14:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9ef7fe700 1 -- 172.21.15.67:0/2004378567 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7ff9dc000bc0 con 0x7ff9d8032630 2024-02-10T08:14:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2qLHsGv5ND2r9rKza8qLoFsMGgHOnFwJ7N/hJ/8RMVHPuILMYdVnFZLfvvJ6HTEf50hU/WBppqDZvRQyU5Bvi/lRGhm1zlpmiJPg2kXjO7OB8/tzVpPZit/pmVfwyRT17mI33jNk8CY23VUTWhY2ypWvqpj6S86+yH5OIUH2Ab8jKlMfshCVVn6h7yDhLaJNW+UJ1yamibC4XUv5oOPsvMT9fcUaQkHqjHwYpLskVQSgIg8ffgz/54ihXU/MEm6nwhAuGBYmHvYkEzmEama++NuVl3+ARLGDDiT3VFUGQbUh54DJhUJyKiugyldYSJOk3DjIJ60Z6/Gdf10C/dUItfi87TwyxowuylQzZp0euRDmRMzNOpxBKSqKxaGhSsaHC6LQCukzEEBsfQCKLYYM4+h7Qy4X2JD/voodCDYLL1xQ6ttlep0+9+Sg6tGoval2oUxywZTs2kLyRZp/Lsx6BmG7upIaj1rF5zm1Qx+nFAoj4CGUq5DB3E/xLopExM+E= ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:14:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff9d8032630 msgr2=0x7ff9d8034af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff9d8032630 0x7ff9d8034af0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7ff9e4009780 tx=0x7ff9e4006cc0).stop 2024-02-10T08:14:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 msgr2=0x7ff9f010c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:23.803 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f010c4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ff9e8014f80 tx=0x7ff9e8009840).stop 2024-02-10T08:14:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 shutdown_connections 2024-02-10T08:14:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7ff9d8032630 0x7ff9d8034af0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 --2- 172.21.15.67:0/2004378567 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff9f00fd8d0 0x7ff9f010c4c0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0x7ff9e8014f80 tx=0x7ff9e8009840).stop 2024-02-10T08:14:23.804 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.795+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 >> 172.21.15.67:0/2004378567 conn(0x7ff9f00710b0 msgr2=0x7ff9f00fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.799+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 shutdown_connections 2024-02-10T08:14:23.805 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:23.799+0000 7ff9f83a3700 1 -- 172.21.15.67:0/2004378567 wait complete. 2024-02-10T08:14:24.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:24 smithi067 bash[17385]: audit 2024-02-10T08:14:23.799727+0000 mgr.y (mgr.14116) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-10T08:14:24.470 INFO:teuthology.orchestra.run.smithi067.stderr:Adding key to root@localhost authorized_keys... 2024-02-10T08:14:24.471 INFO:teuthology.orchestra.run.smithi067.stderr:Adding host smithi067... 2024-02-10T08:14:25.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e74b9d700 1 Processor -- start 2024-02-10T08:14:25.428 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e74b9d700 1 -- start start 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e74b9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e700fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e74b9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700fbfd0 con 0x7f1e700fb6c0 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e700fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e700fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42820/0 (socket says 172.21.15.67:42820) 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6e59c700 1 -- 172.21.15.67:0/2099435211 learned_addr learned my addr 172.21.15.67:0/2099435211 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6e59c700 1 -- 172.21.15.67:0/2099435211 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700fc110 con 0x7f1e700fb6c0 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6e59c700 1 --2- 172.21.15.67:0/2099435211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e700fbae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1e60009730 tx=0x7f1e600090f0).ready entity=mon.0 client_cookie=653bbce2d4efbb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6d59a700 1 -- 172.21.15.67:0/2099435211 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e6000b580 con 0x7f1e700fb6c0 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6d59a700 1 -- 172.21.15.67:0/2099435211 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f1e6000bb50 con 0x7f1e700fb6c0 2024-02-10T08:14:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.423+0000 7f1e6d59a700 1 -- 172.21.15.67:0/2099435211 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e60011a70 con 0x7f1e700fb6c0 2024-02-10T08:14:25.431 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- 172.21.15.67:0/2099435211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 msgr2=0x7f1e700fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:25.431 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 --2- 172.21.15.67:0/2099435211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e700fbae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1e60009730 tx=0x7f1e600090f0).stop 2024-02-10T08:14:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- 172.21.15.67:0/2099435211 shutdown_connections 2024-02-10T08:14:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 --2- 172.21.15.67:0/2099435211 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e700fbae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- 172.21.15.67:0/2099435211 >> 172.21.15.67:0/2099435211 conn(0x7f1e700f6ec0 msgr2=0x7f1e700f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- 172.21.15.67:0/2099435211 shutdown_connections 2024-02-10T08:14:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- 172.21.15.67:0/2099435211 wait complete. 2024-02-10T08:14:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 Processor -- start 2024-02-10T08:14:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- start start 2024-02-10T08:14:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e7010a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7010a7b0 con 0x7f1e700fb6c0 2024-02-10T08:14:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e6e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e7010a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e6e59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e7010a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42832/0 (socket says 172.21.15.67:42832) 2024-02-10T08:14:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e6e59c700 1 -- 172.21.15.67:0/3495115222 learned_addr learned my addr 172.21.15.67:0/3495115222 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e6e59c700 1 -- 172.21.15.67:0/3495115222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009370 con 0x7f1e700fb6c0 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e6e59c700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e7010a2c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1e60006940 tx=0x7f1e600090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e6001a640 con 0x7f1e700fb6c0 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f1e6001ac10 con 0x7f1e700fb6c0 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7010a9b0 con 0x7f1e700fb6c0 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.427+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e60023b90 con 0x7f1e700fb6c0 2024-02-10T08:14:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7010ae40 con 0x7f1e700fb6c0 2024-02-10T08:14:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7005a3b0 con 0x7f1e700fb6c0 2024-02-10T08:14:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f1e6001a7a0 con 0x7f1e700fb6c0 2024-02-10T08:14:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e5f7fe700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f1e580325c0 0x7f1e58034a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f1e60044fe0 con 0x7f1e700fb6c0 2024-02-10T08:14:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e6dd9b700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f1e580325c0 0x7f1e58034a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.431+0000 7f1e6dd9b700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f1e580325c0 0x7f1e58034a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f1e64009780 tx=0x7f1e64006cb0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:25.442 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.439+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1e60018320 con 0x7f1e700fb6c0 2024-02-10T08:14:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:25.775+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e7010b760 con 0x7f1e580325c0 2024-02-10T08:14:26.076 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:25 smithi067 bash[17647]: Warning: Permanently added '172.21.15.67' (ECDSA) to the list of known hosts. 2024-02-10T08:14:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:26 smithi067 bash[17385]: audit 2024-02-10T08:14:25.782494+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:26.640 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.635+0000 7f1e5f7fe700 1 -- 172.21.15.67:0/3495115222 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f1e7010b760 con 0x7f1e580325c0 2024-02-10T08:14:26.640 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: Added host 'smithi067' with addr '172.21.15.67' 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f1e580325c0 msgr2=0x7f1e58034a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f1e580325c0 0x7f1e58034a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f1e64009780 tx=0x7f1e64006cb0).stop 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 msgr2=0x7f1e7010a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e7010a2c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1e60006940 tx=0x7f1e600090f0).stop 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 shutdown_connections 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f1e580325c0 0x7f1e58034a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:26.645 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 --2- 172.21.15.67:0/3495115222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e700fb6c0 0x7f1e7010a2c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:26.646 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 >> 172.21.15.67:0/3495115222 conn(0x7f1e700f6ec0 msgr2=0x7f1e700f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:26.646 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 shutdown_connections 2024-02-10T08:14:26.646 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:26.639+0000 7f1e74b9d700 1 -- 172.21.15.67:0/3495115222 wait complete. 2024-02-10T08:14:27.500 INFO:teuthology.orchestra.run.smithi067.stderr:Deploying unmanaged mon service... 2024-02-10T08:14:27.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:27 smithi067 bash[17385]: audit 2024-02-10T08:14:26.638911+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:27.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:27 smithi067 bash[17385]: cephadm 2024-02-10T08:14:26.639563+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Added host smithi067 2024-02-10T08:14:27.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:27 smithi067 bash[17385]: audit 2024-02-10T08:14:26.666337+0000 mon.a (mon.0) 54 : audit [DBG] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:29.240 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 Processor -- start 2024-02-10T08:14:29.240 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 -- start start 2024-02-10T08:14:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b40fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40fbfd0 con 0x7f84b40fb6c0 2024-02-10T08:14:29.241 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b8ad7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b40fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b8ad7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b40fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42840/0 (socket says 172.21.15.67:42840) 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b8ad7700 1 -- 172.21.15.67:0/3320840651 learned_addr learned my addr 172.21.15.67:0/3320840651 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b8ad7700 1 -- 172.21.15.67:0/3320840651 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40fc110 con 0x7f84b40fb6c0 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b8ad7700 1 --2- 172.21.15.67:0/3320840651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b40fbae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f84a4009730 tx=0x7f84a40090f0).ready entity=mon.0 client_cookie=2138772cff4f1f62 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b37fe700 1 -- 172.21.15.67:0/3320840651 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a400b580 con 0x7f84b40fb6c0 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b37fe700 1 -- 172.21.15.67:0/3320840651 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f84a400bb50 con 0x7f84b40fb6c0 2024-02-10T08:14:29.242 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84b37fe700 1 -- 172.21.15.67:0/3320840651 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a4011a70 con 0x7f84b40fb6c0 2024-02-10T08:14:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 -- 172.21.15.67:0/3320840651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 msgr2=0x7f84b40fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 --2- 172.21.15.67:0/3320840651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b40fbae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f84a4009730 tx=0x7f84a40090f0).stop 2024-02-10T08:14:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 -- 172.21.15.67:0/3320840651 shutdown_connections 2024-02-10T08:14:29.243 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 --2- 172.21.15.67:0/3320840651 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b40fbae0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.235+0000 7f84bad3b700 1 -- 172.21.15.67:0/3320840651 >> 172.21.15.67:0/3320840651 conn(0x7f84b40f6ec0 msgr2=0x7f84b40f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84bad3b700 1 -- 172.21.15.67:0/3320840651 shutdown_connections 2024-02-10T08:14:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84bad3b700 1 -- 172.21.15.67:0/3320840651 wait complete. 2024-02-10T08:14:29.244 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84bad3b700 1 Processor -- start 2024-02-10T08:14:29.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84bad3b700 1 -- start start 2024-02-10T08:14:29.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84bad3b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b410a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:29.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84bad3b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b410a7b0 con 0x7f84b40fb6c0 2024-02-10T08:14:29.245 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84b8ad7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b410a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84b8ad7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b410a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42852/0 (socket says 172.21.15.67:42852) 2024-02-10T08:14:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84b8ad7700 1 -- 172.21.15.67:0/74727288 learned_addr learned my addr 172.21.15.67:0/74727288 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84b8ad7700 1 -- 172.21.15.67:0/74727288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a4009370 con 0x7f84b40fb6c0 2024-02-10T08:14:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.239+0000 7f84b8ad7700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b410a2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f84a4009f00 tx=0x7f84a40090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:29.246 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a401a760 con 0x7f84b40fb6c0 2024-02-10T08:14:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b410a9b0 con 0x7f84b40fb6c0 2024-02-10T08:14:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1010+0+0 (secure 0 0 0) 0x7f84a401ad30 con 0x7f84b40fb6c0 2024-02-10T08:14:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84a4023bd0 con 0x7f84b40fb6c0 2024-02-10T08:14:29.247 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b410ae40 con 0x7f84b40fb6c0 2024-02-10T08:14:29.248 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b405a3b0 con 0x7f84b40fb6c0 2024-02-10T08:14:29.248 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f84a4023410 con 0x7f84b40fb6c0 2024-02-10T08:14:29.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b1ffb700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f849c032630 0x7f849c034af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:29.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f84a4045010 con 0x7f84b40fb6c0 2024-02-10T08:14:29.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b3fff700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f849c032630 0x7f849c034af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:29.249 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.243+0000 7f84b3fff700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f849c032630 0x7f849c034af0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f84a8009780 tx=0x7f84a8006cb0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:29.253 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.247+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f84a40199e0 con 0x7f84b40fb6c0 2024-02-10T08:14:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.603+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f84b410b760 con 0x7f849c032630 2024-02-10T08:14:29.615 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84b1ffb700 1 -- 172.21.15.67:0/74727288 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f84b410b760 con 0x7f849c032630 2024-02-10T08:14:29.615 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-10T08:14:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f849c032630 msgr2=0x7f849c034af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f849c032630 0x7f849c034af0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f84a8009780 tx=0x7f84a8006cb0).stop 2024-02-10T08:14:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 msgr2=0x7f84b410a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b410a2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f84a4009f00 tx=0x7f84a40090f0).stop 2024-02-10T08:14:29.618 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 shutdown_connections 2024-02-10T08:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f849c032630 0x7f849c034af0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 --2- 172.21.15.67:0/74727288 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84b40fb6c0 0x7f84b410a2c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 >> 172.21.15.67:0/74727288 conn(0x7f84b40f6ec0 msgr2=0x7f84b40f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 shutdown_connections 2024-02-10T08:14:29.619 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:29.611+0000 7f84bad3b700 1 -- 172.21.15.67:0/74727288 wait complete. 2024-02-10T08:14:30.319 INFO:teuthology.orchestra.run.smithi067.stderr:Deploying unmanaged mgr service... 2024-02-10T08:14:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:30 smithi067 bash[17385]: audit 2024-02-10T08:14:29.608361+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:30 smithi067 bash[17385]: cephadm 2024-02-10T08:14:29.611582+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-10T08:14:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:30 smithi067 bash[17385]: audit 2024-02-10T08:14:29.614900+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:30 smithi067 bash[17385]: audit 2024-02-10T08:14:29.906150+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:30 smithi067 bash[17385]: audit 2024-02-10T08:14:30.162759+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:31.263 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f0965a4b700 1 Processor -- start 2024-02-10T08:14:31.263 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f0965a4b700 1 -- start start 2024-02-10T08:14:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f0965a4b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f09600fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:31.264 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f0965a4b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09600fc620 con 0x7f09600fbc40 2024-02-10T08:14:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f09600fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f09600fc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42862/0 (socket says 172.21.15.67:42862) 2024-02-10T08:14:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095effd700 1 -- 172.21.15.67:0/3107953731 learned_addr learned my addr 172.21.15.67:0/3107953731 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095effd700 1 -- 172.21.15.67:0/3107953731 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600fc760 con 0x7f09600fbc40 2024-02-10T08:14:31.265 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095effd700 1 --2- 172.21.15.67:0/3107953731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f09600fc060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0950009e50 tx=0x7f0950009840).ready entity=mon.0 client_cookie=c2954a5b24376763 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:31.266 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095dffb700 1 -- 172.21.15.67:0/3107953731 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095000e070 con 0x7f09600fbc40 2024-02-10T08:14:31.266 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095dffb700 1 -- 172.21.15.67:0/3107953731 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f0950013420 con 0x7f09600fbc40 2024-02-10T08:14:31.266 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.259+0000 7f095dffb700 1 -- 172.21.15.67:0/3107953731 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0950007400 con 0x7f09600fbc40 2024-02-10T08:14:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/3107953731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 msgr2=0x7f09600fc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 --2- 172.21.15.67:0/3107953731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f09600fc060 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f0950009e50 tx=0x7f0950009840).stop 2024-02-10T08:14:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/3107953731 shutdown_connections 2024-02-10T08:14:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 --2- 172.21.15.67:0/3107953731 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f09600fc060 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/3107953731 >> 172.21.15.67:0/3107953731 conn(0x7f09600f5460 msgr2=0x7f09600f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/3107953731 shutdown_connections 2024-02-10T08:14:31.268 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/3107953731 wait complete. 2024-02-10T08:14:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 Processor -- start 2024-02-10T08:14:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- start start 2024-02-10T08:14:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f096010c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096010c710 con 0x7f09600fbc40 2024-02-10T08:14:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f095effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f096010c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f095effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f096010c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42876/0 (socket says 172.21.15.67:42876) 2024-02-10T08:14:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f095effd700 1 -- 172.21.15.67:0/2174745232 learned_addr learned my addr 172.21.15.67:0/2174745232 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f095effd700 1 -- 172.21.15.67:0/2174745232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950009ae0 con 0x7f09600fbc40 2024-02-10T08:14:31.270 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f095effd700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f096010c220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f0950000f80 tx=0x7f0950009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095000e040 con 0x7f09600fbc40 2024-02-10T08:14:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f0950012d60 con 0x7f09600fbc40 2024-02-10T08:14:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0950007bc0 con 0x7f09600fbc40 2024-02-10T08:14:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096010c910 con 0x7f09600fbc40 2024-02-10T08:14:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.263+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096010cda0 con 0x7f09600fbc40 2024-02-10T08:14:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.267+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f0950003040 con 0x7f09600fbc40 2024-02-10T08:14:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.267+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960046d80 con 0x7f09600fbc40 2024-02-10T08:14:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.267+0000 7f0957fff700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f094c032610 0x7f094c034ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:31.272 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.267+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f09500440d0 con 0x7f09600fbc40 2024-02-10T08:14:31.273 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.267+0000 7f095e7fc700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f094c032610 0x7f094c034ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:31.273 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.267+0000 7f095e7fc700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f094c032610 0x7f094c034ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0948009780 tx=0x7f0948006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:31.278 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.271+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0950013e60 con 0x7f09600fbc40 2024-02-10T08:14:31.630 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.623+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f09600f7680 con 0x7f094c032610 2024-02-10T08:14:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.631+0000 7f0957fff700 1 -- 172.21.15.67:0/2174745232 <== mgr.14116 v2:172.21.15.67:6800/2933691832 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f09600f7680 con 0x7f094c032610 2024-02-10T08:14:31.639 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-10T08:14:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f094c032610 msgr2=0x7f094c034ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:31.643 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f094c032610 0x7f094c034ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f0948009780 tx=0x7f0948006cc0).stop 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 msgr2=0x7f096010c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f096010c220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f0950000f80 tx=0x7f0950009840).stop 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 shutdown_connections 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f094c032610 0x7f094c034ad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 --2- 172.21.15.67:0/2174745232 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f09600fbc40 0x7f096010c220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.635+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 >> 172.21.15.67:0/2174745232 conn(0x7f09600f5460 msgr2=0x7f09600fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.639+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 shutdown_connections 2024-02-10T08:14:31.644 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:31.639+0000 7f0965a4b700 1 -- 172.21.15.67:0/2174745232 wait complete. 2024-02-10T08:14:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:32 smithi067 bash[17385]: audit 2024-02-10T08:14:31.632021+0000 mgr.y (mgr.14116) 13 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:32 smithi067 bash[17385]: cephadm 2024-02-10T08:14:31.634355+0000 mgr.y (mgr.14116) 14 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-10T08:14:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:32 smithi067 bash[17385]: audit 2024-02-10T08:14:31.637887+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.335+0000 7f4cce54e700 1 Processor -- start 2024-02-10T08:14:33.344 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- start start 2024-02-10T08:14:33.349 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc80faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80faf80 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc80faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc80faa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42888/0 (socket says 172.21.15.67:42888) 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc7fff700 1 -- 172.21.15.67:0/3357154639 learned_addr learned my addr 172.21.15.67:0/3357154639 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc7fff700 1 -- 172.21.15.67:0/3357154639 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80fb0c0 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc7fff700 1 --2- 172.21.15.67:0/3357154639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc80faa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4cb8009730 tx=0x7f4cb80090f0).ready entity=mon.0 client_cookie=5e712f1537d10430 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc6ffd700 1 -- 172.21.15.67:0/3357154639 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb800b580 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc6ffd700 1 -- 172.21.15.67:0/3357154639 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f4cb800bb70 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cc6ffd700 1 -- 172.21.15.67:0/3357154639 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb8011a90 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- 172.21.15.67:0/3357154639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 msgr2=0x7f4cc80faa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 --2- 172.21.15.67:0/3357154639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc80faa00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4cb8009730 tx=0x7f4cb80090f0).stop 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- 172.21.15.67:0/3357154639 shutdown_connections 2024-02-10T08:14:33.350 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 --2- 172.21.15.67:0/3357154639 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc80faa00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- 172.21.15.67:0/3357154639 >> 172.21.15.67:0/3357154639 conn(0x7f4cc80f5d00 msgr2=0x7f4cc80f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- 172.21.15.67:0/3357154639 shutdown_connections 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.339+0000 7f4cce54e700 1 -- 172.21.15.67:0/3357154639 wait complete. 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cce54e700 1 Processor -- start 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cce54e700 1 -- start start 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cce54e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc810a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cce54e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc810a780 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc810a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc7fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc810a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42896/0 (socket says 172.21.15.67:42896) 2024-02-10T08:14:33.351 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc7fff700 1 -- 172.21.15.67:0/2992152311 learned_addr learned my addr 172.21.15.67:0/2992152311 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc7fff700 1 -- 172.21.15.67:0/2992152311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb8009370 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc7fff700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc810a290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4cb80092b0 tx=0x7f4cb80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb801a640 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f4cb801ac30 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4cb8022900 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc810a980 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.352 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.343+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc810ae10 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.347+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f4cb8022b30 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.347+0000 7f4cc57fa700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f4cb00325e0 0x7f4cb0034aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.347+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f4cb8044a10 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.347+0000 7f4cc77fe700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f4cb00325e0 0x7f4cb0034aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.347+0000 7f4cc77fe700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f4cb00325e0 0x7f4cb0034aa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4cbc00ad90 tx=0x7f4cbc009410).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:33.353 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.347+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc805a3b0 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.359 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.355+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4cb801e070 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.692 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.687+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f4cc80fc560 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.695 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cc57fa700 1 -- 172.21.15.67:0/2992152311 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f4cb8019390 con 0x7f4cc80fa5e0 2024-02-10T08:14:33.699 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f4cb00325e0 msgr2=0x7f4cb0034aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f4cb00325e0 0x7f4cb0034aa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4cbc00ad90 tx=0x7f4cbc009410).stop 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 msgr2=0x7f4cc810a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc810a290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4cb80092b0 tx=0x7f4cb80090f0).stop 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 shutdown_connections 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f4cb00325e0 0x7f4cb0034aa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 --2- 172.21.15.67:0/2992152311 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4cc80fa5e0 0x7f4cc810a290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.691+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 >> 172.21.15.67:0/2992152311 conn(0x7f4cc80f5d00 msgr2=0x7f4cc80f6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:33.700 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.695+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 shutdown_connections 2024-02-10T08:14:33.701 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:33.695+0000 7f4cce54e700 1 -- 172.21.15.67:0/2992152311 wait complete. 2024-02-10T08:14:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:34 smithi067 bash[17385]: audit 2024-02-10T08:14:33.695603+0000 mon.a (mon.0) 59 : audit [INF] from='client.? 172.21.15.67:0/2992152311' entity='client.admin' 2024-02-10T08:14:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.383+0000 7fa623ce8700 1 Processor -- start 2024-02-10T08:14:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.383+0000 7fa623ce8700 1 -- start start 2024-02-10T08:14:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c0fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:35.391 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0fc590 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa621a84700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c0fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:35.392 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa621a84700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c0fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59164/0 (socket says 172.21.15.67:59164) 2024-02-10T08:14:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa621a84700 1 -- 172.21.15.67:0/3813084375 learned_addr learned my addr 172.21.15.67:0/3813084375 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa621a84700 1 -- 172.21.15.67:0/3813084375 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0fc6d0 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa621a84700 1 --2- 172.21.15.67:0/3813084375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c0fc010 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa618009730 tx=0x7fa6180090f0).ready entity=mon.0 client_cookie=4dd82e7e204940de server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa620a82700 1 -- 172.21.15.67:0/3813084375 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa61800b580 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa620a82700 1 -- 172.21.15.67:0/3813084375 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa61800bb70 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.393 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa620a82700 1 -- 172.21.15.67:0/3813084375 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa618011a90 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 -- 172.21.15.67:0/3813084375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 msgr2=0x7fa61c0fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 --2- 172.21.15.67:0/3813084375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c0fc010 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fa618009730 tx=0x7fa6180090f0).stop 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 -- 172.21.15.67:0/3813084375 shutdown_connections 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 --2- 172.21.15.67:0/3813084375 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c0fc010 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 -- 172.21.15.67:0/3813084375 >> 172.21.15.67:0/3813084375 conn(0x7fa61c0f5460 msgr2=0x7fa61c0f78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 -- 172.21.15.67:0/3813084375 shutdown_connections 2024-02-10T08:14:35.394 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.387+0000 7fa623ce8700 1 -- 172.21.15.67:0/3813084375 wait complete. 2024-02-10T08:14:35.395 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa623ce8700 1 Processor -- start 2024-02-10T08:14:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa623ce8700 1 -- start start 2024-02-10T08:14:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa623ce8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c10c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa623ce8700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c10c910 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa621a84700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c10c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa621a84700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c10c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59168/0 (socket says 172.21.15.67:59168) 2024-02-10T08:14:35.396 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa621a84700 1 -- 172.21.15.67:0/3139419103 learned_addr learned my addr 172.21.15.67:0/3139419103 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa621a84700 1 -- 172.21.15.67:0/3139419103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa618009370 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa621a84700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c10c420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa618011440 tx=0x7fa6180090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa618011cd0 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa618011e30 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c10cb10 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c10cfa0 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.397 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa61801a6d0 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.398 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.391+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7fa61801a900 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.398 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.395+0000 7fa612ffd700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa6080325e0 0x7fa608034aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.395+0000 7fa621283700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa6080325e0 0x7fa608034aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.395+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa600004f50 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.395+0000 7fa621283700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa6080325e0 0x7fa608034aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fa60c0097d0 tx=0x7fa60c006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:35.399 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.395+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa6180457f0 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.407 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.403+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa618021540 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.739 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.731+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa600005780 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.745 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.739+0000 7fa612ffd700 1 -- 172.21.15.67:0/3139419103 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa618021360 con 0x7fa61c0fbbf0 2024-02-10T08:14:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa6080325e0 msgr2=0x7fa608034aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa6080325e0 0x7fa608034aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fa60c0097d0 tx=0x7fa60c006cc0).stop 2024-02-10T08:14:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 msgr2=0x7fa61c10c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:35.750 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c10c420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa618011440 tx=0x7fa6180090f0).stop 2024-02-10T08:14:35.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 shutdown_connections 2024-02-10T08:14:35.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa6080325e0 0x7fa608034aa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:35.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 --2- 172.21.15.67:0/3139419103 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa61c0fbbf0 0x7fa61c10c420 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0x7fa618011440 tx=0x7fa6180090f0).stop 2024-02-10T08:14:35.751 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 >> 172.21.15.67:0/3139419103 conn(0x7fa61c0f5460 msgr2=0x7fa61c0f6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 shutdown_connections 2024-02-10T08:14:35.752 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:35.743+0000 7fa623ce8700 1 -- 172.21.15.67:0/3139419103 wait complete. 2024-02-10T08:14:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:35 smithi067 bash[17385]: cluster 2024-02-10T08:14:34.853752+0000 mgr.y (mgr.14116) 15 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:14:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:35 smithi067 bash[17385]: audit 2024-02-10T08:14:35.213666+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14116 172.21.15.67:0/122721905' entity='mgr.y' 2024-02-10T08:14:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:35 smithi067 bash[17385]: audit 2024-02-10T08:14:35.744867+0000 mon.a (mon.0) 61 : audit [INF] from='client.? 172.21.15.67:0/3139419103' entity='client.admin' 2024-02-10T08:14:36.487 INFO:teuthology.orchestra.run.smithi067.stderr:Enabling the dashboard module... 2024-02-10T08:14:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035ff48700 1 Processor -- start 2024-02-10T08:14:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035ff48700 1 -- start start 2024-02-10T08:14:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035ff48700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:37.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035ff48700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580fa370 con 0x7f03580f99d0 2024-02-10T08:14:37.451 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035dce4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035dce4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59176/0 (socket says 172.21.15.67:59176) 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.443+0000 7f035dce4700 1 -- 172.21.15.67:0/2393275805 learned_addr learned my addr 172.21.15.67:0/2393275805 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035dce4700 1 -- 172.21.15.67:0/2393275805 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03580fa4b0 con 0x7f03580f99d0 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035dce4700 1 --2- 172.21.15.67:0/2393275805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580f9df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0348009e50 tx=0x7f0348009840).ready entity=mon.0 client_cookie=fc599a5ecbe91172 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035cce2700 1 -- 172.21.15.67:0/2393275805 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f034800e070 con 0x7f03580f99d0 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035cce2700 1 -- 172.21.15.67:0/2393275805 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f0348013420 con 0x7f03580f99d0 2024-02-10T08:14:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035cce2700 1 -- 172.21.15.67:0/2393275805 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0348007400 con 0x7f03580f99d0 2024-02-10T08:14:37.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 -- 172.21.15.67:0/2393275805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 msgr2=0x7f03580f9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:37.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 --2- 172.21.15.67:0/2393275805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580f9df0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f0348009e50 tx=0x7f0348009840).stop 2024-02-10T08:14:37.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 -- 172.21.15.67:0/2393275805 shutdown_connections 2024-02-10T08:14:37.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 --2- 172.21.15.67:0/2393275805 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580f9df0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:37.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 -- 172.21.15.67:0/2393275805 >> 172.21.15.67:0/2393275805 conn(0x7f03580f5460 msgr2=0x7f03580f78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 -- 172.21.15.67:0/2393275805 shutdown_connections 2024-02-10T08:14:37.454 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.447+0000 7f035ff48700 1 -- 172.21.15.67:0/2393275805 wait complete. 2024-02-10T08:14:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035ff48700 1 Processor -- start 2024-02-10T08:14:37.455 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035ff48700 1 -- start start 2024-02-10T08:14:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035ff48700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035ff48700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03580fa9b0 con 0x7f03580f99d0 2024-02-10T08:14:37.456 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035dce4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035dce4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59178/0 (socket says 172.21.15.67:59178) 2024-02-10T08:14:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035dce4700 1 -- 172.21.15.67:0/3064338851 learned_addr learned my addr 172.21.15.67:0/3064338851 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:37.457 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035dce4700 1 -- 172.21.15.67:0/3064338851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348009ae0 con 0x7f03580f99d0 2024-02-10T08:14:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035dce4700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580fc530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f03480139f0 tx=0x7f0348009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0348013ea0 con 0x7f03580f99d0 2024-02-10T08:14:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f0348007b00 con 0x7f03580f99d0 2024-02-10T08:14:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f034800e040 con 0x7f03580f99d0 2024-02-10T08:14:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03580fab50 con 0x7f03580f99d0 2024-02-10T08:14:37.458 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.451+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03580fafe0 con 0x7f03580f99d0 2024-02-10T08:14:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.455+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 8) v1 ==== 38617+0+0 (secure 0 0 0) 0x7f0348003040 con 0x7f03580f99d0 2024-02-10T08:14:37.459 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.455+0000 7f0356ffd700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f03400325e0 0x7f0340034aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:37.460 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.455+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035805a3b0 con 0x7f03580f99d0 2024-02-10T08:14:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.455+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7f03480440b0 con 0x7f03580f99d0 2024-02-10T08:14:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.455+0000 7f035d4e3700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f03400325e0 0x7f0340034aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:37.464 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.455+0000 7f035d4e3700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f03400325e0 0x7f0340034aa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f034c009780 tx=0x7f034c006cc0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:37.465 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.459+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f0348024540 con 0x7f03580f99d0 2024-02-10T08:14:37.882 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.875+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f0358046d80 con 0x7f03580f99d0 2024-02-10T08:14:37.971 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.963+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 9) v1 ==== 38630+0+0 (secure 0 0 0) 0x7f0348005c00 con 0x7f03580f99d0 2024-02-10T08:14:37.973 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.967+0000 7f0356ffd700 1 -- 172.21.15.67:0/3064338851 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f0348012cb0 con 0x7f03580f99d0 2024-02-10T08:14:37.977 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f03400325e0 msgr2=0x7f0340034aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f03400325e0 0x7f0340034aa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f034c009780 tx=0x7f034c006cc0).stop 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 msgr2=0x7f03580fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580fc530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f03480139f0 tx=0x7f0348009840).stop 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 shutdown_connections 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7f03400325e0 0x7f0340034aa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 --2- 172.21.15.67:0/3064338851 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f03580f99d0 0x7f03580fc530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 >> 172.21.15.67:0/3064338851 conn(0x7f03580f5460 msgr2=0x7f03580f6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 shutdown_connections 2024-02-10T08:14:37.978 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:37.971+0000 7f035ff48700 1 -- 172.21.15.67:0/3064338851 wait complete. 2024-02-10T08:14:38.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:37 smithi067 bash[17385]: cluster 2024-02-10T08:14:36.854149+0000 mgr.y (mgr.14116) 16 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:14:38.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:37 smithi067 bash[17385]: audit 2024-02-10T08:14:37.882225+0000 mon.a (mon.0) 62 : audit [INF] from='client.? 172.21.15.67:0/3064338851' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-10T08:14:38.249 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17647]: ignoring --setuser ceph since I am not root 2024-02-10T08:14:38.249 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17647]: ignoring --setgroup ceph since I am not root 2024-02-10T08:14:38.249 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17647]: debug 2024-02-10T08:14:38.163+0000 7f964d592680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T08:14:38.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17647]: debug 2024-02-10T08:14:38.263+0000 7f964d592680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T08:14:38.975 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17647]: debug 2024-02-10T08:14:38.715+0000 7f964d592680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T08:14:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17385]: audit 2024-02-10T08:14:37.973007+0000 mon.a (mon.0) 63 : audit [INF] from='client.? 172.21.15.67:0/3064338851' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-10T08:14:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:38 smithi067 bash[17385]: cluster 2024-02-10T08:14:37.973122+0000 mon.a (mon.0) 64 : cluster [DBG] mgrmap e9: y(active, since 23s) 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb14aef3700 1 Processor -- start 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb14aef3700 1 -- start start 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb14aef3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb1440702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb14aef3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb144072120 con 0x7fb144071d00 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb148c8f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb1440702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb148c8f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb1440702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59194/0 (socket says 172.21.15.67:59194) 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.891+0000 7fb148c8f700 1 -- 172.21.15.67:0/4200776115 learned_addr learned my addr 172.21.15.67:0/4200776115 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb148c8f700 1 -- 172.21.15.67:0/4200776115 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144070790 con 0x7fb144071d00 2024-02-10T08:14:39.899 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb148c8f700 1 --2- 172.21.15.67:0/4200776115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb1440702a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb138009730 tx=0x7fb1380090f0).ready entity=mon.0 client_cookie=cf3243c91d3ad81f server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb1437fe700 1 -- 172.21.15.67:0/4200776115 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb13800b580 con 0x7fb144071d00 2024-02-10T08:14:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb1437fe700 1 -- 172.21.15.67:0/4200776115 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fb13800bb70 con 0x7fb144071d00 2024-02-10T08:14:39.900 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb1437fe700 1 -- 172.21.15.67:0/4200776115 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb138011a90 con 0x7fb144071d00 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 -- 172.21.15.67:0/4200776115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 msgr2=0x7fb1440702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 --2- 172.21.15.67:0/4200776115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb1440702a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fb138009730 tx=0x7fb1380090f0).stop 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 -- 172.21.15.67:0/4200776115 shutdown_connections 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 --2- 172.21.15.67:0/4200776115 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb1440702a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 -- 172.21.15.67:0/4200776115 >> 172.21.15.67:0/4200776115 conn(0x7fb1440f6eb0 msgr2=0x7fb1440f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 -- 172.21.15.67:0/4200776115 shutdown_connections 2024-02-10T08:14:39.901 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 -- 172.21.15.67:0/4200776115 wait complete. 2024-02-10T08:14:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 Processor -- start 2024-02-10T08:14:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.895+0000 7fb14aef3700 1 -- start start 2024-02-10T08:14:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb14aef3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb14410c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:39.902 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb14aef3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14410c9c0 con 0x7fb144071d00 2024-02-10T08:14:39.903 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb148c8f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb14410c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:39.903 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb148c8f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb14410c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59206/0 (socket says 172.21.15.67:59206) 2024-02-10T08:14:39.903 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb148c8f700 1 -- 172.21.15.67:0/1667544861 learned_addr learned my addr 172.21.15.67:0/1667544861 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:39.904 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb148c8f700 1 -- 172.21.15.67:0/1667544861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb138009370 con 0x7fb144071d00 2024-02-10T08:14:39.904 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb148c8f700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb14410c4d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fb1380092b0 tx=0x7fb1380090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb13801a640 con 0x7fb144071d00 2024-02-10T08:14:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fb13801ac30 con 0x7fb144071d00 2024-02-10T08:14:39.906 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14410cbc0 con 0x7fb144071d00 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14410d050 con 0x7fb144071d00 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb138022900 con 0x7fb144071d00 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) v1 ==== 38630+0+0 (secure 0 0 0) 0x7fb138022b40 con 0x7fb144071d00 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb141ffb700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 0x7fb134034850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fb1380449a0 con 0x7fb144071d00 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb143fff700 1 -- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 msgr2=0x7fb134034850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2933691832 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb143fff700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 0x7fb134034850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:14:39.907 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.899+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14405a3b0 con 0x7fb144071d00 2024-02-10T08:14:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:39.911+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb138018d80 con 0x7fb144071d00 2024-02-10T08:14:40.107 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.103+0000 7fb143fff700 1 -- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 msgr2=0x7fb134034850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2933691832 2024-02-10T08:14:40.107 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.103+0000 7fb143fff700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 0x7fb134034850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-10T08:14:40.107 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:39 smithi067 bash[17647]: debug 2024-02-10T08:14:39.723+0000 7f964d592680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T08:14:40.108 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:39 smithi067 bash[17647]: debug 2024-02-10T08:14:39.831+0000 7f964d592680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T08:14:40.334 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.327+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb1441019c0 con 0x7fb144071d00 2024-02-10T08:14:40.334 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.327+0000 7fb141ffb700 1 -- 172.21.15.67:0/1667544861 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fb138018320 con 0x7fb144071d00 2024-02-10T08:14:40.334 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:14:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-10T08:14:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "available": true, 2024-02-10T08:14:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-10T08:14:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-10T08:14:40.336 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:14:40.337 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 msgr2=0x7fb134034850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:14:40.338 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 0x7fb134034850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:40.339 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 msgr2=0x7fb14410c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb14410c4d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fb1380092b0 tx=0x7fb1380090f0).stop 2024-02-10T08:14:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 shutdown_connections 2024-02-10T08:14:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fb134032390 0x7fb134034850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 --2- 172.21.15.67:0/1667544861 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb144071d00 0x7fb14410c4d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 >> 172.21.15.67:0/1667544861 conn(0x7fb1440f6eb0 msgr2=0x7fb1440f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 shutdown_connections 2024-02-10T08:14:40.341 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:40.331+0000 7fb14aef3700 1 -- 172.21.15.67:0/1667544861 wait complete. 2024-02-10T08:14:40.368 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:40 smithi067 bash[17647]: debug 2024-02-10T08:14:40.103+0000 7f964d592680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T08:14:40.619 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:40 smithi067 bash[17385]: audit 2024-02-10T08:14:40.334265+0000 mon.a (mon.0) 65 : audit [DBG] from='client.? 172.21.15.67:0/1667544861' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T08:14:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for the mgr to restart... 2024-02-10T08:14:41.022 INFO:teuthology.orchestra.run.smithi067.stderr:Waiting for mgr epoch 9... 2024-02-10T08:14:41.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:41 smithi067 bash[17647]: debug 2024-02-10T08:14:41.291+0000 7f964d592680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T08:14:41.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:41 smithi067 bash[17647]: debug 2024-02-10T08:14:41.463+0000 7f964d592680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:14:41.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:41 smithi067 bash[17647]: debug 2024-02-10T08:14:41.567+0000 7f964d592680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T08:14:41.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:41 smithi067 bash[17647]: debug 2024-02-10T08:14:41.679+0000 7f964d592680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T08:14:42.295 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:41 smithi067 bash[17647]: debug 2024-02-10T08:14:41.907+0000 7f964d592680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T08:14:42.619 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:42 smithi067 bash[17647]: debug 2024-02-10T08:14:42.291+0000 7f964d592680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T08:14:42.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:42 smithi067 bash[17647]: debug 2024-02-10T08:14:42.411+0000 7f964d592680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T08:14:42.620 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5abf3b700 1 Processor -- start 2024-02-10T08:14:42.620 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5abf3b700 1 -- start start 2024-02-10T08:14:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5abf3b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a40fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:42.621 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5abf3b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40fc560 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a9cd7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a40fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a9cd7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a40fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59222/0 (socket says 172.21.15.67:59222) 2024-02-10T08:14:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a9cd7700 1 -- 172.21.15.67:0/3821447880 learned_addr learned my addr 172.21.15.67:0/3821447880 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a9cd7700 1 -- 172.21.15.67:0/3821447880 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40fc6a0 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.622 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a9cd7700 1 --2- 172.21.15.67:0/3821447880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a40fbfe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa5a000b9d0 tx=0x7fa5a000b3b0).ready entity=mon.0 client_cookie=bb69b9494662f92d server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a8cd5700 1 -- 172.21.15.67:0/3821447880 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5a000da70 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.623 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a8cd5700 1 -- 172.21.15.67:0/3821447880 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa5a000dbd0 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5a8cd5700 1 -- 172.21.15.67:0/3821447880 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5a000f3d0 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5abf3b700 1 -- 172.21.15.67:0/3821447880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 msgr2=0x7fa5a40fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.615+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/3821447880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a40fbfe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa5a000b9d0 tx=0x7fa5a000b3b0).stop 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- 172.21.15.67:0/3821447880 shutdown_connections 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/3821447880 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a40fbfe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- 172.21.15.67:0/3821447880 >> 172.21.15.67:0/3821447880 conn(0x7fa5a40f5460 msgr2=0x7fa5a40f78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- 172.21.15.67:0/3821447880 shutdown_connections 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- 172.21.15.67:0/3821447880 wait complete. 2024-02-10T08:14:42.624 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 Processor -- start 2024-02-10T08:14:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- start start 2024-02-10T08:14:42.625 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a41080c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a41085b0 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5a9cd7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a41080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5a9cd7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a41080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59236/0 (socket says 172.21.15.67:59236) 2024-02-10T08:14:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5a9cd7700 1 -- 172.21.15.67:0/4189058126 learned_addr learned my addr 172.21.15.67:0/4189058126 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5a9cd7700 1 -- 172.21.15.67:0/4189058126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a000b610 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5a9cd7700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a41080c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa5a00109e0 tx=0x7fa5a000b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5a001e070 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa5a0023420 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a41087b0 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5a0023e80 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.619+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4108c40 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.623+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 9) v1 ==== 38630+0+0 (secure 0 0 0) 0x7fa5a0019af0 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.623+0000 7fa59affd700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.623+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 --> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa590035190 con 0x7fa590032610 2024-02-10T08:14:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.623+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 926+0+0 (secure 0 0 0) 0x7fa5a0047010 con 0x7fa5a40fbbc0 2024-02-10T08:14:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.623+0000 7fa5a94d6700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 msgr2=0x7fa590034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2933691832 2024-02-10T08:14:42.628 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.623+0000 7fa5a94d6700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:14:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.827+0000 7fa5a94d6700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 msgr2=0x7fa590034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2933691832 2024-02-10T08:14:42.831 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:42.827+0000 7fa5a94d6700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-10T08:14:43.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:43.231+0000 7fa5a94d6700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 msgr2=0x7fa590034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2933691832 2024-02-10T08:14:43.235 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:43.231+0000 7fa5a94d6700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-10T08:14:43.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:43 smithi067 bash[17647]: debug 2024-02-10T08:14:43.331+0000 7f964d592680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T08:14:43.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:43 smithi067 bash[17647]: debug 2024-02-10T08:14:43.435+0000 7f964d592680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T08:14:43.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:43 smithi067 bash[17647]: debug 2024-02-10T08:14:43.559+0000 7f964d592680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T08:14:44.039 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:44.035+0000 7fa5a94d6700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 msgr2=0x7fa590034ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/2933691832 2024-02-10T08:14:44.039 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:44.035+0000 7fa5a94d6700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-10T08:14:44.089 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:43 smithi067 bash[17647]: debug 2024-02-10T08:14:43.783+0000 7f964d592680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T08:14:44.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17647]: debug 2024-02-10T08:14:44.083+0000 7f964d592680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T08:14:44.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17647]: debug 2024-02-10T08:14:44.251+0000 7f964d592680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T08:14:44.771 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17647]: debug 2024-02-10T08:14:44.431+0000 7f964d592680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:14:44.771 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17647]: debug 2024-02-10T08:14:44.651+0000 7f964d592680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T08:14:44.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:44.795+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mgrmap(e 10) v1 ==== 38454+0+0 (secure 0 0 0) 0x7fa5a002d440 con 0x7fa5a40fbbc0 2024-02-10T08:14:44.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:44.795+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 msgr2=0x7fa590034ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:14:44.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:44.795+0000 7fa59affd700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:45.030 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17647]: debug 2024-02-10T08:14:44.763+0000 7f964d592680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T08:14:45.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: cluster 2024-02-10T08:14:44.793079+0000 mon.a (mon.0) 66 : cluster [INF] Active manager daemon y restarted 2024-02-10T08:14:45.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: cluster 2024-02-10T08:14:44.793538+0000 mon.a (mon.0) 67 : cluster [INF] Activating manager daemon y 2024-02-10T08:14:45.030 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: cluster 2024-02-10T08:14:44.801226+0000 mon.a (mon.0) 68 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: cluster 2024-02-10T08:14:44.801416+0000 mon.a (mon.0) 69 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00805332s) 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: audit 2024-02-10T08:14:44.802432+0000 mon.a (mon.0) 70 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: audit 2024-02-10T08:14:44.803736+0000 mon.a (mon.0) 71 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: audit 2024-02-10T08:14:44.804905+0000 mon.a (mon.0) 72 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: audit 2024-02-10T08:14:44.805108+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: audit 2024-02-10T08:14:44.805309+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T08:14:45.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:44 smithi067 bash[17385]: cluster 2024-02-10T08:14:44.809225+0000 mon.a (mon.0) 75 : cluster [INF] Manager daemon y is now available 2024-02-10T08:14:45.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:14:45 smithi067 bash[17647]: Warning: Permanently added '172.21.15.67' (ECDSA) to the list of known hosts. 2024-02-10T08:14:45.801 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.795+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mgrmap(e 11) v1 ==== 38524+0+0 (secure 0 0 0) 0x7fa5a000fa70 con 0x7fa5a40fbbc0 2024-02-10T08:14:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.795+0000 7fa59affd700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa590036190 0x7fa590038580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.795+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa590035190 con 0x7fa590036190 2024-02-10T08:14:45.802 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.795+0000 7fa5a94d6700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa590036190 0x7fa590038580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:45.803 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.799+0000 7fa5a94d6700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa590036190 0x7fa590038580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fa594003850 tx=0x7fa594006e60).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:45.805 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.799+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7fa590035190 con 0x7fa590036190 2024-02-10T08:14:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa5a405a3b0 con 0x7fa590036190 2024-02-10T08:14:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa59affd700 1 -- 172.21.15.67:0/4189058126 <== mgr.14146 v2:172.21.15.67:6800/4022632212 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa5a405a3b0 con 0x7fa590036190 2024-02-10T08:14:45.817 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: { 2024-02-10T08:14:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-10T08:14:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: "initialized": true 2024-02-10T08:14:45.818 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: } 2024-02-10T08:14:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa590036190 msgr2=0x7fa590038580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa590036190 0x7fa590038580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7fa594003850 tx=0x7fa594006e60).stop 2024-02-10T08:14:45.819 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 msgr2=0x7fa5a41080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a41080c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa5a00109e0 tx=0x7fa5a000b3b0).stop 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 shutdown_connections 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa590036190 0x7fa590038580 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa5a40fbbc0 0x7fa5a41080c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 --2- 172.21.15.67:0/4189058126 >> [v2:172.21.15.67:6800/2933691832,v1:172.21.15.67:6801/2933691832] conn(0x7fa590032610 0x7fa590034ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 >> 172.21.15.67:0/4189058126 conn(0x7fa5a40f5460 msgr2=0x7fa5a40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 shutdown_connections 2024-02-10T08:14:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:45.811+0000 7fa5abf3b700 1 -- 172.21.15.67:0/4189058126 wait complete. 2024-02-10T08:14:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:45 smithi067 bash[17385]: audit 2024-02-10T08:14:44.957511+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:45 smithi067 bash[17385]: audit 2024-02-10T08:14:44.960538+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:14:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:45 smithi067 bash[17385]: audit 2024-02-10T08:14:45.005487+0000 mon.a (mon.0) 78 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:14:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:45 smithi067 bash[17385]: audit 2024-02-10T08:14:45.011440+0000 mon.a (mon.0) 79 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:14:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:45 smithi067 bash[17385]: cluster 2024-02-10T08:14:45.804471+0000 mon.a (mon.0) 80 : cluster [DBG] mgrmap e11: y(active, since 1.0111s) 2024-02-10T08:14:46.478 INFO:teuthology.orchestra.run.smithi067.stderr:mgr epoch 9 is available 2024-02-10T08:14:46.478 INFO:teuthology.orchestra.run.smithi067.stderr:Generating a dashboard self-signed certificate... 2024-02-10T08:14:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:46 smithi067 bash[17385]: audit 2024-02-10T08:14:45.804126+0000 mgr.y (mgr.14146) 1 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T08:14:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:46 smithi067 bash[17385]: audit 2024-02-10T08:14:45.817264+0000 mgr.y (mgr.14146) 2 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T08:14:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 Processor -- start 2024-02-10T08:14:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 -- start start 2024-02-10T08:14:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec0fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0fe270 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.444 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34eb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec0fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34eb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec0fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44904/0 (socket says 172.21.15.67:44904) 2024-02-10T08:14:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34eb7fe700 1 -- 172.21.15.67:0/3430493554 learned_addr learned my addr 172.21.15.67:0/3430493554 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34eb7fe700 1 -- 172.21.15.67:0/3430493554 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0fe3b0 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.445 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34eb7fe700 1 --2- 172.21.15.67:0/3430493554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec0fdcf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f34dc009730 tx=0x7f34dc0090f0).ready entity=mon.0 client_cookie=8823ecbc3c88f988 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34ea7fc700 1 -- 172.21.15.67:0/3430493554 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34dc00b580 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34ea7fc700 1 -- 172.21.15.67:0/3430493554 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f34dc00bb70 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.446 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34ea7fc700 1 -- 172.21.15.67:0/3430493554 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34dc011a90 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3430493554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 msgr2=0x7f34ec0fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 --2- 172.21.15.67:0/3430493554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec0fdcf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f34dc009730 tx=0x7f34dc0090f0).stop 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3430493554 shutdown_connections 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.439+0000 7f34f1aaa700 1 --2- 172.21.15.67:0/3430493554 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec0fdcf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3430493554 >> 172.21.15.67:0/3430493554 conn(0x7f34ec070e40 msgr2=0x7f34ec071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3430493554 shutdown_connections 2024-02-10T08:14:47.447 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3430493554 wait complete. 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 Processor -- start 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- start start 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec10c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec10c980 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34eb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec10c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34eb7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec10c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44920/0 (socket says 172.21.15.67:44920) 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34eb7fe700 1 -- 172.21.15.67:0/3917170785 learned_addr learned my addr 172.21.15.67:0/3917170785 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:47.449 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34eb7fe700 1 -- 172.21.15.67:0/3917170785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc009370 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34eb7fe700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec10c490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f34dc0092b0 tx=0x7f34dc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34dc01a640 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f34dc01ac30 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34dc022900 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec10cb80 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.443+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec10d010 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.451 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.447+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 38630+0+0 (secure 0 0 0) 0x7f34dc01a7a0 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.447+0000 7f34e8f74700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f34d4032680 0x7f34d4034b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.447+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec046d80 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.447+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f34dc044a40 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.452 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.447+0000 7f34eaffd700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f34d4032680 0x7f34d4034b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:47.453 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.447+0000 7f34eaffd700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f34d4032680 0x7f34d4034b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f34e0009780 tx=0x7f34e0006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:47.459 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.455+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f34dc01e080 con 0x7f34ec0fd8d0 2024-02-10T08:14:47.800 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.795+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f34ec10d870 con 0x7f34d4032680 2024-02-10T08:14:47.866 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.859+0000 7f34e8f74700 1 -- 172.21.15.67:0/3917170785 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f34ec10d870 con 0x7f34d4032680 2024-02-10T08:14:47.866 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f34d4032680 msgr2=0x7f34d4034b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f34d4032680 0x7f34d4034b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f34e0009780 tx=0x7f34e0006cc0).stop 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 msgr2=0x7f34ec10c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec10c490 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f34dc0092b0 tx=0x7f34dc0090f0).stop 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 shutdown_connections 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f34d4032680 0x7f34d4034b40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 --2- 172.21.15.67:0/3917170785 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f34ec0fd8d0 0x7f34ec10c490 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 >> 172.21.15.67:0/3917170785 conn(0x7f34ec070e40 msgr2=0x7f34ec0fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 shutdown_connections 2024-02-10T08:14:47.870 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:47.863+0000 7f34f1aaa700 1 -- 172.21.15.67:0/3917170785 wait complete. 2024-02-10T08:14:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:47 smithi067 bash[17385]: cluster 2024-02-10T08:14:46.851145+0000 mon.a (mon.0) 81 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-02-10T08:14:48.607 INFO:teuthology.orchestra.run.smithi067.stderr:Creating initial admin user... 2024-02-10T08:14:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:48 smithi067 bash[17385]: audit 2024-02-10T08:14:47.801015+0000 mgr.y (mgr.14146) 3 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:48 smithi067 bash[17385]: audit 2024-02-10T08:14:47.862518+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:14:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:48 smithi067 bash[17385]: audit 2024-02-10T08:14:47.865482+0000 mon.a (mon.0) 83 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:14:49.556 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd320cb1700 1 Processor -- start 2024-02-10T08:14:49.557 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd320cb1700 1 -- start start 2024-02-10T08:14:49.557 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd320cb1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c0fdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:49.557 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd320cb1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c0fe270 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.557 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c0fdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c0fdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44922/0 (socket says 172.21.15.67:44922) 2024-02-10T08:14:49.558 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31a59c700 1 -- 172.21.15.67:0/3920317393 learned_addr learned my addr 172.21.15.67:0/3920317393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31a59c700 1 -- 172.21.15.67:0/3920317393 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c0fe3b0 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31a59c700 1 --2- 172.21.15.67:0/3920317393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c0fdcf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fd304009740 tx=0x7fd304009060).ready entity=mon.0 client_cookie=68b26be190cb549 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31959a700 1 -- 172.21.15.67:0/3920317393 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30400b520 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31959a700 1 -- 172.21.15.67:0/3920317393 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fd30400bb10 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.559 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.551+0000 7fd31959a700 1 -- 172.21.15.67:0/3920317393 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3040119c0 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- 172.21.15.67:0/3920317393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 msgr2=0x7fd31c0fdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 --2- 172.21.15.67:0/3920317393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c0fdcf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fd304009740 tx=0x7fd304009060).stop 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- 172.21.15.67:0/3920317393 shutdown_connections 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 --2- 172.21.15.67:0/3920317393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c0fdcf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- 172.21.15.67:0/3920317393 >> 172.21.15.67:0/3920317393 conn(0x7fd31c0710b0 msgr2=0x7fd31c0714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- 172.21.15.67:0/3920317393 shutdown_connections 2024-02-10T08:14:49.560 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- 172.21.15.67:0/3920317393 wait complete. 2024-02-10T08:14:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 Processor -- start 2024-02-10T08:14:49.561 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- start start 2024-02-10T08:14:49.562 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c10c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:49.562 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd320cb1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31c10c960 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.562 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd31a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c10c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:49.562 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd31a59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c10c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44924/0 (socket says 172.21.15.67:44924) 2024-02-10T08:14:49.562 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.555+0000 7fd31a59c700 1 -- 172.21.15.67:0/1192541577 learned_addr learned my addr 172.21.15.67:0/1192541577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:49.563 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd31a59c700 1 -- 172.21.15.67:0/1192541577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304009300 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.563 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd31a59c700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c10c470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd304011440 tx=0x7fd304009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd30401a400 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fd30401a9f0 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31c10cb60 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd304019890 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31c10cff0 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.564 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 12) v1 ==== 38630+0+0 (secure 0 0 0) 0x7fd30401a560 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.565 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd3137fe700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd308032660 0x7fd308034b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:49.565 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fd304044cd0 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.565 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd319d9b700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd308032660 0x7fd308034b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:49.566 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31c05a3b0 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.566 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.559+0000 7fd319d9b700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd308032660 0x7fd308034b20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd30c009780 tx=0x7fd30c006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:49.573 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.567+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd304019b00 con 0x7fd31c0fd8d0 2024-02-10T08:14:49.934 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:49.927+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd31c046d80 con 0x7fd308032660 2024-02-10T08:14:50.213 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.207+0000 7fd3137fe700 1 -- 172.21.15.67:0/1192541577 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd31c046d80 con 0x7fd308032660 2024-02-10T08:14:50.214 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$NFQ.MlKLWHsCIYT4W6MiSOCVdZCSPdrjC8I2QoFKCd7NlPlvcYqm.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707552890, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-10T08:14:50.216 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd308032660 msgr2=0x7fd308034b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:50.216 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd308032660 0x7fd308034b20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fd30c009780 tx=0x7fd30c006cc0).stop 2024-02-10T08:14:50.216 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 msgr2=0x7fd31c10c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c10c470 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd304011440 tx=0x7fd304009060).stop 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 shutdown_connections 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd308032660 0x7fd308034b20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 --2- 172.21.15.67:0/1192541577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd31c0fd8d0 0x7fd31c10c470 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 >> 172.21.15.67:0/1192541577 conn(0x7fd31c0710b0 msgr2=0x7fd31c0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 shutdown_connections 2024-02-10T08:14:50.217 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:50.211+0000 7fd320cb1700 1 -- 172.21.15.67:0/1192541577 wait complete. 2024-02-10T08:14:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:50 smithi067 bash[17385]: audit 2024-02-10T08:14:49.934528+0000 mgr.y (mgr.14146) 4 : audit [DBG] from='client.14158 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:14:50.930 INFO:teuthology.orchestra.run.smithi067.stderr:Fetching dashboard port number... 2024-02-10T08:14:51.619 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:51 smithi067 bash[17385]: audit 2024-02-10T08:14:50.212541+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:14:51.913 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2f101700 1 Processor -- start 2024-02-10T08:14:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2f101700 1 -- start start 2024-02-10T08:14:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2f101700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2f101700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280fe860 con 0x7f5e2806fca0 2024-02-10T08:14:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2ce9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2ce9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44940/0 (socket says 172.21.15.67:44940) 2024-02-10T08:14:51.914 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2ce9d700 1 -- 172.21.15.67:0/4150552161 learned_addr learned my addr 172.21.15.67:0/4150552161 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:51.915 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2ce9d700 1 -- 172.21.15.67:0/4150552161 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e280fe9a0 con 0x7f5e2806fca0 2024-02-10T08:14:51.915 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.907+0000 7f5e2ce9d700 1 --2- 172.21.15.67:0/4150552161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fe2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f5e24009730 tx=0x7f5e240090f0).ready entity=mon.0 client_cookie=17853c547a6307b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:51.915 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e1f7fe700 1 -- 172.21.15.67:0/4150552161 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e2400b580 con 0x7f5e2806fca0 2024-02-10T08:14:51.915 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e1f7fe700 1 -- 172.21.15.67:0/4150552161 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f5e2400bb70 con 0x7f5e2806fca0 2024-02-10T08:14:51.915 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e1f7fe700 1 -- 172.21.15.67:0/4150552161 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e24011a90 con 0x7f5e2806fca0 2024-02-10T08:14:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- 172.21.15.67:0/4150552161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 msgr2=0x7f5e280fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:51.916 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 --2- 172.21.15.67:0/4150552161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fe2e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f5e24009730 tx=0x7f5e240090f0).stop 2024-02-10T08:14:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- 172.21.15.67:0/4150552161 shutdown_connections 2024-02-10T08:14:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 --2- 172.21.15.67:0/4150552161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fe2e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- 172.21.15.67:0/4150552161 >> 172.21.15.67:0/4150552161 conn(0x7f5e280f5d00 msgr2=0x7f5e280f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- 172.21.15.67:0/4150552161 shutdown_connections 2024-02-10T08:14:51.917 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- 172.21.15.67:0/4150552161 wait complete. 2024-02-10T08:14:51.918 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 Processor -- start 2024-02-10T08:14:51.918 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- start start 2024-02-10T08:14:51.918 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:51.918 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2f101700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e280fa1e0 con 0x7f5e2806fca0 2024-02-10T08:14:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2ce9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2ce9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44956/0 (socket says 172.21.15.67:44956) 2024-02-10T08:14:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.911+0000 7f5e2ce9d700 1 -- 172.21.15.67:0/3349152030 learned_addr learned my addr 172.21.15.67:0/3349152030 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e2ce9d700 1 -- 172.21.15.67:0/3349152030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24009370 con 0x7f5e2806fca0 2024-02-10T08:14:51.919 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e2ce9d700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fbd60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f5e24009f00 tx=0x7f5e240090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e24011cd0 con 0x7f5e2806fca0 2024-02-10T08:14:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e280fa380 con 0x7f5e2806fca0 2024-02-10T08:14:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f5e24011e30 con 0x7f5e2806fca0 2024-02-10T08:14:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e280fa810 con 0x7f5e2806fca0 2024-02-10T08:14:51.920 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e2401a5f0 con 0x7f5e2806fca0 2024-02-10T08:14:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f5e2401a860 con 0x7f5e2806fca0 2024-02-10T08:14:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1dffb700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5e10032750 0x7f5e10034c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f5e24044ca0 con 0x7f5e2806fca0 2024-02-10T08:14:51.921 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1ffff700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5e10032750 0x7f5e10034c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e14004f20 con 0x7f5e2806fca0 2024-02-10T08:14:51.922 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.915+0000 7f5e1ffff700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5e10032750 0x7f5e10034c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5e18009780 tx=0x7f5e18006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:51.928 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:51.923+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f5e2401e080 con 0x7f5e2806fca0 2024-02-10T08:14:52.257 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.251+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f5e14005750 con 0x7f5e2806fca0 2024-02-10T08:14:52.257 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.251+0000 7f5e1dffb700 1 -- 172.21.15.67:0/3349152030 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f5e24021360 con 0x7f5e2806fca0 2024-02-10T08:14:52.258 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 8443 2024-02-10T08:14:52.260 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5e10032750 msgr2=0x7f5e10034c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5e10032750 0x7f5e10034c10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f5e18009780 tx=0x7f5e18006cc0).stop 2024-02-10T08:14:52.261 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 msgr2=0x7f5e280fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fbd60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f5e24009f00 tx=0x7f5e240090f0).stop 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 shutdown_connections 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5e10032750 0x7f5e10034c10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 --2- 172.21.15.67:0/3349152030 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f5e2806fca0 0x7f5e280fbd60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 >> 172.21.15.67:0/3349152030 conn(0x7f5e280f5d00 msgr2=0x7f5e280f7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 shutdown_connections 2024-02-10T08:14:52.262 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:52.255+0000 7f5e2f101700 1 -- 172.21.15.67:0/3349152030 wait complete. 2024-02-10T08:14:52.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:52 smithi067 bash[17385]: cluster 2024-02-10T08:14:51.219469+0000 mon.a (mon.0) 85 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-02-10T08:14:52.497 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:52 smithi067 bash[17385]: audit 2024-02-10T08:14:51.671830+0000 mon.a (mon.0) 86 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:14:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:firewalld does not appear to be present 2024-02-10T08:14:53.003 INFO:teuthology.orchestra.run.smithi067.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-10T08:14:53.006 INFO:teuthology.orchestra.run.smithi067.stderr:Ceph Dashboard is now available at: 2024-02-10T08:14:53.006 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:53.007 INFO:teuthology.orchestra.run.smithi067.stderr: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-02-10T08:14:53.007 INFO:teuthology.orchestra.run.smithi067.stderr: User: admin 2024-02-10T08:14:53.007 INFO:teuthology.orchestra.run.smithi067.stderr: Password: ri2lm7ixfx 2024-02-10T08:14:53.007 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:53.007 INFO:teuthology.orchestra.run.smithi067.stderr:Enabling autotune for osd_memory_target 2024-02-10T08:14:53.369 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:53 smithi067 bash[17385]: audit 2024-02-10T08:14:52.257401+0000 mon.a (mon.0) 87 : audit [DBG] from='client.? 172.21.15.67:0/3349152030' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc99cac700 1 Processor -- start 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc99cac700 1 -- start start 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc99cac700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc940fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc99cac700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc940fbfc0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc940fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc940fbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38182/0 (socket says 172.21.15.67:38182) 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc937fe700 1 -- 172.21.15.67:0/1922338625 learned_addr learned my addr 172.21.15.67:0/1922338625 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc937fe700 1 -- 172.21.15.67:0/1922338625 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940fc100 con 0x7fbc940fb6b0 2024-02-10T08:14:54.018 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc937fe700 1 --2- 172.21.15.67:0/1922338625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc940fbad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fbc84009730 tx=0x7fbc840090f0).ready entity=mon.0 client_cookie=26eef45b081dde8a server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.011+0000 7fbc927fc700 1 -- 172.21.15.67:0/1922338625 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc8400b580 con 0x7fbc940fb6b0 2024-02-10T08:14:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc927fc700 1 -- 172.21.15.67:0/1922338625 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fbc8400bb70 con 0x7fbc940fb6b0 2024-02-10T08:14:54.019 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc927fc700 1 -- 172.21.15.67:0/1922338625 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc84011a90 con 0x7fbc940fb6b0 2024-02-10T08:14:54.020 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/1922338625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 msgr2=0x7fbc940fbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:54.020 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 --2- 172.21.15.67:0/1922338625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc940fbad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fbc84009730 tx=0x7fbc840090f0).stop 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/1922338625 shutdown_connections 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 --2- 172.21.15.67:0/1922338625 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc940fbad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/1922338625 >> 172.21.15.67:0/1922338625 conn(0x7fbc940f6eb0 msgr2=0x7fbc940f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/1922338625 shutdown_connections 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/1922338625 wait complete. 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 Processor -- start 2024-02-10T08:14:54.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- start start 2024-02-10T08:14:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc9410a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9410a6e0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc9410a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc937fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc9410a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38194/0 (socket says 172.21.15.67:38194) 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc937fe700 1 -- 172.21.15.67:0/3864232796 learned_addr learned my addr 172.21.15.67:0/3864232796 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc937fe700 1 -- 172.21.15.67:0/3864232796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84009370 con 0x7fbc940fb6b0 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc937fe700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc9410a1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fbc84000c00 tx=0x7fbc840090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc8401a7b0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fbc8401ada0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.023 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9410a8e0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc84023c50 con 0x7fbc940fb6b0 2024-02-10T08:14:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.015+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9410ad70 con 0x7fbc940fb6b0 2024-02-10T08:14:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.019+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fbc8401a910 con 0x7fbc940fb6b0 2024-02-10T08:14:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.019+0000 7fbc90ff9700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fbc7c032720 0x7fbc7c034be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.019+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc94046d80 con 0x7fbc940fb6b0 2024-02-10T08:14:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.019+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fbc84045220 con 0x7fbc940fb6b0 2024-02-10T08:14:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.019+0000 7fbc92ffd700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fbc7c032720 0x7fbc7c034be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:54.025 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.019+0000 7fbc92ffd700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fbc7c032720 0x7fbc7c034be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbc88009780 tx=0x7fbc88006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:54.031 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.027+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbc8401e020 con 0x7fbc940fb6b0 2024-02-10T08:14:54.357 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.351+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fbc940fd4b0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.358 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.351+0000 7fbc90ff9700 1 -- 172.21.15.67:0/3864232796 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fbc840199e0 con 0x7fbc940fb6b0 2024-02-10T08:14:54.362 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fbc7c032720 msgr2=0x7fbc7c034be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:54.362 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fbc7c032720 0x7fbc7c034be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fbc88009780 tx=0x7fbc88006cc0).stop 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 msgr2=0x7fbc9410a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc9410a1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fbc84000c00 tx=0x7fbc840090f0).stop 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 shutdown_connections 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fbc7c032720 0x7fbc7c034be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 --2- 172.21.15.67:0/3864232796 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbc940fb6b0 0x7fbc9410a1f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.355+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 >> 172.21.15.67:0/3864232796 conn(0x7fbc940f6eb0 msgr2=0x7fbc940f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.359+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 shutdown_connections 2024-02-10T08:14:54.364 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:54.359+0000 7fbc99cac700 1 -- 172.21.15.67:0/3864232796 wait complete. 2024-02-10T08:14:56.475 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 Processor -- start 2024-02-10T08:14:56.476 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- start start 2024-02-10T08:14:56.476 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c0fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c0fbfd0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c5affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c0fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c5affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c0fbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38202/0 (socket says 172.21.15.67:38202) 2024-02-10T08:14:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c5affd700 1 -- 172.21.15.67:0/1130796470 learned_addr learned my addr 172.21.15.67:0/1130796470 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:56.477 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c5affd700 1 -- 172.21.15.67:0/1130796470 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c0fc110 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c5affd700 1 --2- 172.21.15.67:0/1130796470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c0fbae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7c4c009730 tx=0x7f7c4c0090f0).ready entity=mon.0 client_cookie=1cb26195138014ce server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c59ffb700 1 -- 172.21.15.67:0/1130796470 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c4c00b580 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c59ffb700 1 -- 172.21.15.67:0/1130796470 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f7c4c00bb70 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c59ffb700 1 -- 172.21.15.67:0/1130796470 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c4c011a90 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- 172.21.15.67:0/1130796470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 msgr2=0x7f7c5c0fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:56.478 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 --2- 172.21.15.67:0/1130796470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c0fbae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7c4c009730 tx=0x7f7c4c0090f0).stop 2024-02-10T08:14:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- 172.21.15.67:0/1130796470 shutdown_connections 2024-02-10T08:14:56.479 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 --2- 172.21.15.67:0/1130796470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c0fbae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- 172.21.15.67:0/1130796470 >> 172.21.15.67:0/1130796470 conn(0x7f7c5c0f6ec0 msgr2=0x7f7c5c0f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- 172.21.15.67:0/1130796470 shutdown_connections 2024-02-10T08:14:56.480 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 -- 172.21.15.67:0/1130796470 wait complete. 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.471+0000 7f7c61340700 1 Processor -- start 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c61340700 1 -- start start 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c61340700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c10a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c61340700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c10a6f0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c5affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c10a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c5affd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c10a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38206/0 (socket says 172.21.15.67:38206) 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c5affd700 1 -- 172.21.15.67:0/1159953807 learned_addr learned my addr 172.21.15.67:0/1159953807 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c5affd700 1 -- 172.21.15.67:0/1159953807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c009370 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c5affd700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c10a200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7c4c000c00 tx=0x7f7c4c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c4c01a7b0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f7c4c01ada0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.482 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c10a8f0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c4c023c50 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c10ad80 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f7c4c01a910 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.475+0000 7f7c43fff700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7c44032720 0x7f7c44034be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:14:56.483 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.479+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c046d80 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.484 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.479+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f7c4c045220 con 0x7f7c5c0fb6c0 2024-02-10T08:14:56.484 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.479+0000 7f7c5a7fc700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7c44032720 0x7f7c44034be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:14:56.484 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.479+0000 7f7c5a7fc700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7c44032720 0x7f7c44034be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f7c50009780 tx=0x7f7c50006cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:14:56.487 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:56.483+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7c4c01e020 con 0x7f7c5c0fb6c0 2024-02-10T08:14:57.013 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.007+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f7c5c0fd4c0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:57.016 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.011+0000 7f7c43fff700 1 -- 172.21.15.67:0/1159953807 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v24)=0 set mgr/dashboard/cluster/status v24) v1 ==== 153+0+0 (secure 0 0 0) 0x7f7c4c0199e0 con 0x7f7c5c0fb6c0 2024-02-10T08:14:57.021 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7c44032720 msgr2=0x7f7c44034be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7c44032720 0x7f7c44034be0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f7c50009780 tx=0x7f7c50006cb0).stop 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 msgr2=0x7f7c5c10a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c10a200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7c4c000c00 tx=0x7f7c4c0090f0).stop 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 shutdown_connections 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7c44032720 0x7f7c44034be0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 --2- 172.21.15.67:0/1159953807 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7c5c0fb6c0 0x7f7c5c10a200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 >> 172.21.15.67:0/1159953807 conn(0x7f7c5c0f6ec0 msgr2=0x7f7c5c0f86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 shutdown_connections 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: 2024-02-10T08:14:57.015+0000 7f7c61340700 1 -- 172.21.15.67:0/1159953807 wait complete. 2024-02-10T08:14:57.022 INFO:teuthology.orchestra.run.smithi067.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr: sudo /usr/sbin/cephadm shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr:Or, if you are only running a single cluster on this host: 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.971 INFO:teuthology.orchestra.run.smithi067.stderr: sudo /usr/sbin/cephadm shell 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: ceph telemetry on 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr:For more information see: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:14:57.972 INFO:teuthology.orchestra.run.smithi067.stderr:Bootstrap complete. 2024-02-10T08:14:58.008 INFO:tasks.cephadm:Fetching config... 2024-02-10T08:14:58.009 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:14:58.009 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-10T08:14:58.015 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-10T08:14:58.015 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:14:58.016 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-10T08:14:58.065 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-10T08:14:58.065 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:14:58.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-10T08:14:58.116 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-10T08:14:58.117 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:14:58.117 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-10T08:14:58.165 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-10T08:14:58.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2qLHsGv5ND2r9rKza8qLoFsMGgHOnFwJ7N/hJ/8RMVHPuILMYdVnFZLfvvJ6HTEf50hU/WBppqDZvRQyU5Bvi/lRGhm1zlpmiJPg2kXjO7OB8/tzVpPZit/pmVfwyRT17mI33jNk8CY23VUTWhY2ypWvqpj6S86+yH5OIUH2Ab8jKlMfshCVVn6h7yDhLaJNW+UJ1yamibC4XUv5oOPsvMT9fcUaQkHqjHwYpLskVQSgIg8ffgz/54ihXU/MEm6nwhAuGBYmHvYkEzmEama++NuVl3+ARLGDDiT3VFUGQbUh54DJhUJyKiugyldYSJOk3DjIJ60Z6/Gdf10C/dUItfi87TwyxowuylQzZp0euRDmRMzNOpxBKSqKxaGhSsaHC6LQCukzEEBsfQCKLYYM4+h7Qy4X2JD/voodCDYLL1xQ6ttlep0+9+Sg6tGoval2oUxywZTs2kLyRZp/Lsx6BmG7upIaj1rF5zm1Qx+nFAoj4CGUq5DB3E/xLopExM+E= ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T08:14:58.228 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2qLHsGv5ND2r9rKza8qLoFsMGgHOnFwJ7N/hJ/8RMVHPuILMYdVnFZLfvvJ6HTEf50hU/WBppqDZvRQyU5Bvi/lRGhm1zlpmiJPg2kXjO7OB8/tzVpPZit/pmVfwyRT17mI33jNk8CY23VUTWhY2ypWvqpj6S86+yH5OIUH2Ab8jKlMfshCVVn6h7yDhLaJNW+UJ1yamibC4XUv5oOPsvMT9fcUaQkHqjHwYpLskVQSgIg8ffgz/54ihXU/MEm6nwhAuGBYmHvYkEzmEama++NuVl3+ARLGDDiT3VFUGQbUh54DJhUJyKiugyldYSJOk3DjIJ60Z6/Gdf10C/dUItfi87TwyxowuylQzZp0euRDmRMzNOpxBKSqKxaGhSsaHC6LQCukzEEBsfQCKLYYM4+h7Qy4X2JD/voodCDYLL1xQ6ttlep0+9+Sg6tGoval2oUxywZTs2kLyRZp/Lsx6BmG7upIaj1rF5zm1Qx+nFAoj4CGUq5DB3E/xLopExM+E= ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:14:58.240 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2qLHsGv5ND2r9rKza8qLoFsMGgHOnFwJ7N/hJ/8RMVHPuILMYdVnFZLfvvJ6HTEf50hU/WBppqDZvRQyU5Bvi/lRGhm1zlpmiJPg2kXjO7OB8/tzVpPZit/pmVfwyRT17mI33jNk8CY23VUTWhY2ypWvqpj6S86+yH5OIUH2Ab8jKlMfshCVVn6h7yDhLaJNW+UJ1yamibC4XUv5oOPsvMT9fcUaQkHqjHwYpLskVQSgIg8ffgz/54ihXU/MEm6nwhAuGBYmHvYkEzmEama++NuVl3+ARLGDDiT3VFUGQbUh54DJhUJyKiugyldYSJOk3DjIJ60Z6/Gdf10C/dUItfi87TwyxowuylQzZp0euRDmRMzNOpxBKSqKxaGhSsaHC6LQCukzEEBsfQCKLYYM4+h7Qy4X2JD/voodCDYLL1xQ6ttlep0+9+Sg6tGoval2oUxywZTs2kLyRZp/Lsx6BmG7upIaj1rF5zm1Qx+nFAoj4CGUq5DB3E/xLopExM+E= ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T08:14:58.262 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC2qLHsGv5ND2r9rKza8qLoFsMGgHOnFwJ7N/hJ/8RMVHPuILMYdVnFZLfvvJ6HTEf50hU/WBppqDZvRQyU5Bvi/lRGhm1zlpmiJPg2kXjO7OB8/tzVpPZit/pmVfwyRT17mI33jNk8CY23VUTWhY2ypWvqpj6S86+yH5OIUH2Ab8jKlMfshCVVn6h7yDhLaJNW+UJ1yamibC4XUv5oOPsvMT9fcUaQkHqjHwYpLskVQSgIg8ffgz/54ihXU/MEm6nwhAuGBYmHvYkEzmEama++NuVl3+ARLGDDiT3VFUGQbUh54DJhUJyKiugyldYSJOk3DjIJ60Z6/Gdf10C/dUItfi87TwyxowuylQzZp0euRDmRMzNOpxBKSqKxaGhSsaHC6LQCukzEEBsfQCKLYYM4+h7Qy4X2JD/voodCDYLL1xQ6ttlep0+9+Sg6tGoval2oUxywZTs2kLyRZp/Lsx6BmG7upIaj1rF5zm1Qx+nFAoj4CGUq5DB3E/xLopExM+E= ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:14:58.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-10T08:14:58.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:14:58 smithi067 bash[17385]: audit 2024-02-10T08:14:57.016824+0000 mon.a (mon.0) 88 : audit [INF] from='client.? 172.21.15.67:0/1159953807' entity='client.admin' 2024-02-10T08:15:02.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 -- 172.21.15.67:0/1382125291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 msgr2=0x7f57780f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:02.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 --2- 172.21.15.67:0/1382125291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f57780f60f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f576c004770 tx=0x7f576c009cb0).stop 2024-02-10T08:15:02.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 -- 172.21.15.67:0/1382125291 shutdown_connections 2024-02-10T08:15:02.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 --2- 172.21.15.67:0/1382125291 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f57780f60f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:02.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 -- 172.21.15.67:0/1382125291 >> 172.21.15.67:0/1382125291 conn(0x7f57780f1450 msgr2=0x7f57780f3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:02.158 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 -- 172.21.15.67:0/1382125291 shutdown_connections 2024-02-10T08:15:02.159 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.151+0000 7f577f0b3700 1 -- 172.21.15.67:0/1382125291 wait complete. 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 Processor -- start 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 -- start start 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f5778108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57781094c0 con 0x7f57780f5cd0 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577ce4f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f5778108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577ce4f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f5778108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:38224/0 (socket says 172.21.15.67:38224) 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577ce4f700 1 -- 172.21.15.67:0/2189383551 learned_addr learned my addr 172.21.15.67:0/2189383551 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577ce4f700 1 -- 172.21.15.67:0/2189383551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c005040 con 0x7f57780f5cd0 2024-02-10T08:15:02.160 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577ce4f700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f5778108fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f576c009450 tx=0x7f576c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:02.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f576c00e030 con 0x7f57780f5cd0 2024-02-10T08:15:02.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781096c0 con 0x7f57780f5cd0 2024-02-10T08:15:02.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f576c01d420 con 0x7f57780f5cd0 2024-02-10T08:15:02.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f576c018070 con 0x7f57780f5cd0 2024-02-10T08:15:02.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778109b50 con 0x7f57780f5cd0 2024-02-10T08:15:02.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577805a3b0 con 0x7f57780f5cd0 2024-02-10T08:15:02.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f576c027440 con 0x7f57780f5cd0 2024-02-10T08:15:02.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5775ffb700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f57680326a0 0x7f5768034b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:02.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f576c044400 con 0x7f57780f5cd0 2024-02-10T08:15:02.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.155+0000 7f5777fff700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f57680326a0 0x7f5768034b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:02.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.159+0000 7f5777fff700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f57680326a0 0x7f5768034b60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5764009780 tx=0x7f5764006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:02.170 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.167+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f576c01b760 con 0x7f57780f5cd0 2024-02-10T08:15:02.372 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.367+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f5778046d80 con 0x7f57780f5cd0 2024-02-10T08:15:02.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.371+0000 7f5775ffb700 1 -- 172.21.15.67:0/2189383551 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f576c01b370 con 0x7f57780f5cd0 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f57680326a0 msgr2=0x7f5768034b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f57680326a0 0x7f5768034b60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5764009780 tx=0x7f5764006cc0).stop 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 msgr2=0x7f5778108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f5778108fd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f576c009450 tx=0x7f576c009cb0).stop 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 shutdown_connections 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f57680326a0 0x7f5768034b60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 --2- 172.21.15.67:0/2189383551 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f57780f5cd0 0x7f5778108fd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 >> 172.21.15.67:0/2189383551 conn(0x7f57780f1450 msgr2=0x7f57780f1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 shutdown_connections 2024-02-10T08:15:02.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:02.375+0000 7f577f0b3700 1 -- 172.21.15.67:0/2189383551 wait complete. 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:01.869281+0000 mon.a (mon.0) 89 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.262420+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.268038+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.270275+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.274507+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.376570+0000 mon.a (mon.0) 94 : audit [INF] from='client.? 172.21.15.67:0/2189383551' entity='client.admin' 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.440222+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:15:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:02 smithi067 bash[17385]: audit 2024-02-10T08:15:02.448009+0000 mon.a (mon.0) 96 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:03.458 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-10T08:15:03.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-10T08:15:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:05 smithi067 bash[17385]: cluster 2024-02-10T08:15:04.806914+0000 mgr.y (mgr.14146) 5 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 -- 172.21.15.67:0/1287307669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 msgr2=0x7fb4f00f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 --2- 172.21.15.67:0/1287307669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f00f67d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb4ec004770 tx=0x7fb4ec009cb0).stop 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 -- 172.21.15.67:0/1287307669 shutdown_connections 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 --2- 172.21.15.67:0/1287307669 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f00f67d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 -- 172.21.15.67:0/1287307669 >> 172.21.15.67:0/1287307669 conn(0x7fb4f00f1b10 msgr2=0x7fb4f00f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 -- 172.21.15.67:0/1287307669 shutdown_connections 2024-02-10T08:15:06.910 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 -- 172.21.15.67:0/1287307669 wait complete. 2024-02-10T08:15:06.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 Processor -- start 2024-02-10T08:15:06.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.903+0000 7fb4f672c700 1 -- start start 2024-02-10T08:15:06.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f672c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f0108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:06.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f672c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0109390 con 0x7fb4f00f63b0 2024-02-10T08:15:06.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f572a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f0108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:06.911 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f572a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f0108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54504/0 (socket says 172.21.15.67:54504) 2024-02-10T08:15:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f572a700 1 -- 172.21.15.67:0/1523950891 learned_addr learned my addr 172.21.15.67:0/1523950891 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:15:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f572a700 1 -- 172.21.15.67:0/1523950891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec005040 con 0x7fb4f00f63b0 2024-02-10T08:15:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f572a700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f0108ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb4ec0097f0 tx=0x7fb4ec009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec00e030 con 0x7fb4f00f63b0 2024-02-10T08:15:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fb4ec013e40 con 0x7fb4f00f63b0 2024-02-10T08:15:06.912 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f0109590 con 0x7fb4f00f63b0 2024-02-10T08:15:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4ec018070 con 0x7fb4f00f63b0 2024-02-10T08:15:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f010a140 con 0x7fb4f00f63b0 2024-02-10T08:15:06.913 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fb4ec01c3c0 con 0x7fb4f00f63b0 2024-02-10T08:15:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4e67fc700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fb4dc0323c0 0x7fb4dc034880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.907+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fb4ec049630 con 0x7fb4f00f63b0 2024-02-10T08:15:06.914 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.911+0000 7fb4f4f29700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fb4dc0323c0 0x7fb4dc034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:06.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.911+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f005a3b0 con 0x7fb4f00f63b0 2024-02-10T08:15:06.915 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.911+0000 7fb4f4f29700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fb4dc0323c0 0x7fb4dc034880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fb4e0009780 tx=0x7fb4e0006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:06.922 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:06.919+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb4ec04a060 con 0x7fb4f00f63b0 2024-02-10T08:15:07.112 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.107+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4f00fb5f0 con 0x7fb4dc0323c0 2024-02-10T08:15:07.118 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.115+0000 7fb4e67fc700 1 -- 172.21.15.67:0/1523950891 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb4f00fb5f0 con 0x7fb4dc0323c0 2024-02-10T08:15:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fb4dc0323c0 msgr2=0x7fb4dc034880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:07.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fb4dc0323c0 0x7fb4dc034880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fb4e0009780 tx=0x7fb4e0006cc0).stop 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 msgr2=0x7fb4f0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f0108ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fb4ec0097f0 tx=0x7fb4ec009cb0).stop 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 shutdown_connections 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fb4dc0323c0 0x7fb4dc034880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 --2- 172.21.15.67:0/1523950891 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb4f00f63b0 0x7fb4f0108ea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 >> 172.21.15.67:0/1523950891 conn(0x7fb4f00f1b10 msgr2=0x7fb4f0060de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 shutdown_connections 2024-02-10T08:15:07.123 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:15:07.119+0000 7fb4f672c700 1 -- 172.21.15.67:0/1523950891 wait complete. 2024-02-10T08:15:07.827 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-02-10T08:15:07.827 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:15:07.827 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-02-10T08:15:07.835 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:15:07.835 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:15:07.885 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-02-10T08:15:07.885 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch host add smithi134 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: cluster 2024-02-10T08:15:06.807244+0000 mgr.y (mgr.14146) 6 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: audit 2024-02-10T08:15:07.118263+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: audit 2024-02-10T08:15:07.123929+0000 mon.a (mon.0) 98 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: audit 2024-02-10T08:15:07.126375+0000 mon.a (mon.0) 99 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: audit 2024-02-10T08:15:07.127717+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: audit 2024-02-10T08:15:07.199319+0000 mon.a (mon.0) 101 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:07 smithi067 bash[17385]: audit 2024-02-10T08:15:07.204537+0000 mon.a (mon.0) 102 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:08 smithi067 bash[17385]: cephadm 2024-02-10T08:15:07.129844+0000 mgr.y (mgr.14146) 8 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:15:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:09 smithi067 bash[17385]: cephadm 2024-02-10T08:15:07.148149+0000 mgr.y (mgr.14146) 9 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:15:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:09 smithi067 bash[17385]: cluster 2024-02-10T08:15:08.807618+0000 mgr.y (mgr.14146) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:11 smithi067 bash[17385]: cluster 2024-02-10T08:15:10.808010+0000 mgr.y (mgr.14146) 11 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:13 smithi067 bash[17385]: cluster 2024-02-10T08:15:12.808406+0000 mgr.y (mgr.14146) 12 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:15 smithi067 bash[17385]: cluster 2024-02-10T08:15:14.808820+0000 mgr.y (mgr.14146) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:17 smithi067 bash[17385]: cluster 2024-02-10T08:15:16.809221+0000 mgr.y (mgr.14146) 14 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:19 smithi067 bash[17385]: cluster 2024-02-10T08:15:18.809625+0000 mgr.y (mgr.14146) 15 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:21 smithi067 bash[17385]: cluster 2024-02-10T08:15:20.809958+0000 mgr.y (mgr.14146) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:23 smithi067 bash[17385]: cluster 2024-02-10T08:15:22.810370+0000 mgr.y (mgr.14146) 17 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:25 smithi067 bash[17385]: cluster 2024-02-10T08:15:24.810728+0000 mgr.y (mgr.14146) 18 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:28.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:27 smithi067 bash[17385]: cluster 2024-02-10T08:15:26.811051+0000 mgr.y (mgr.14146) 19 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:29 smithi067 bash[17385]: cluster 2024-02-10T08:15:28.811382+0000 mgr.y (mgr.14146) 20 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:32 smithi067 bash[17385]: cluster 2024-02-10T08:15:30.811827+0000 mgr.y (mgr.14146) 21 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:34 smithi067 bash[17385]: cluster 2024-02-10T08:15:32.812215+0000 mgr.y (mgr.14146) 22 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:36 smithi067 bash[17385]: cluster 2024-02-10T08:15:34.812621+0000 mgr.y (mgr.14146) 23 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:38 smithi067 bash[17385]: cluster 2024-02-10T08:15:36.813029+0000 mgr.y (mgr.14146) 24 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:40 smithi067 bash[17385]: cluster 2024-02-10T08:15:38.813395+0000 mgr.y (mgr.14146) 25 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:42 smithi067 bash[17385]: cluster 2024-02-10T08:15:40.813788+0000 mgr.y (mgr.14146) 26 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:44 smithi067 bash[17385]: cluster 2024-02-10T08:15:42.814214+0000 mgr.y (mgr.14146) 27 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:46 smithi067 bash[17385]: cluster 2024-02-10T08:15:44.814591+0000 mgr.y (mgr.14146) 28 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:48 smithi067 bash[17385]: cluster 2024-02-10T08:15:46.815051+0000 mgr.y (mgr.14146) 29 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:50 smithi067 bash[17385]: cluster 2024-02-10T08:15:48.815422+0000 mgr.y (mgr.14146) 30 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:52 smithi067 bash[17385]: cluster 2024-02-10T08:15:50.815834+0000 mgr.y (mgr.14146) 31 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:54.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:54 smithi067 bash[17385]: cluster 2024-02-10T08:15:52.816231+0000 mgr.y (mgr.14146) 32 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- 172.21.15.134:0/4039184878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 msgr2=0x7ff6700fa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 --2- 172.21.15.134:0/4039184878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff6700fa150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7ff660009ae0 tx=0x7ff660004d10).stop 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- 172.21.15.134:0/4039184878 shutdown_connections 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 --2- 172.21.15.134:0/4039184878 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff6700fa150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- 172.21.15.134:0/4039184878 >> 172.21.15.134:0/4039184878 conn(0x7ff6700f1b10 msgr2=0x7ff6700f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- 172.21.15.134:0/4039184878 shutdown_connections 2024-02-10T08:15:55.057 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- 172.21.15.134:0/4039184878 wait complete. 2024-02-10T08:15:55.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 Processor -- start 2024-02-10T08:15:55.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- start start 2024-02-10T08:15:55.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff670108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:55.058 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff675744700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670109340 con 0x7ff6700611d0 2024-02-10T08:15:55.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff66ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff670108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:55.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff66ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff670108e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.134:56506/0 (socket says 172.21.15.134:56506) 2024-02-10T08:15:55.059 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.052+0000 7ff66ffff700 1 -- 172.21.15.134:0/3988432745 learned_addr learned my addr 172.21.15.134:0/3988432745 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66ffff700 1 -- 172.21.15.134:0/3988432745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6600097c0 con 0x7ff6700611d0 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66ffff700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff670108e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7ff660004b70 tx=0x7ff660004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff66000e030 con 0x7ff6700611d0 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7ff66000bc30 con 0x7ff6700611d0 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff670109540 con 0x7ff6700611d0 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff660018070 con 0x7ff6700611d0 2024-02-10T08:15:55.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67010a150 con 0x7ff6700611d0 2024-02-10T08:15:55.061 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6701099d0 con 0x7ff6700611d0 2024-02-10T08:15:55.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7ff66001d420 con 0x7ff6700611d0 2024-02-10T08:15:55.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66d7fa700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff658032720 0x7ff658034be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:55.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7ff660044650 con 0x7ff6700611d0 2024-02-10T08:15:55.062 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66f7fe700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff658032720 0x7ff658034be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:55.063 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.056+0000 7ff66f7fe700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff658032720 0x7ff658034be0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7ff664009780 tx=0x7ff664006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:55.070 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.064+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff6701099d0 con 0x7ff6700611d0 2024-02-10T08:15:55.258 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:55.252+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6701099d0 con 0x7ff658032720 2024-02-10T08:15:55.620 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:15:55 smithi067 bash[17647]: Warning: Permanently added 'smithi134,172.21.15.134' (ECDSA) to the list of known hosts. 2024-02-10T08:15:56.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.092+0000 7ff66d7fa700 1 -- 172.21.15.134:0/3988432745 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff6701099d0 con 0x7ff658032720 2024-02-10T08:15:56.096 INFO:teuthology.orchestra.run.smithi134.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-02-10T08:15:56.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff658032720 msgr2=0x7ff658034be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:56.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff658032720 0x7ff658034be0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7ff664009780 tx=0x7ff664006cc0).stop 2024-02-10T08:15:56.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 msgr2=0x7ff670108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:56.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff670108e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7ff660004b70 tx=0x7ff660004d10).stop 2024-02-10T08:15:56.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 shutdown_connections 2024-02-10T08:15:56.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff658032720 0x7ff658034be0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:56.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 --2- 172.21.15.134:0/3988432745 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff6700611d0 0x7ff670108e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:56.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 >> 172.21.15.134:0/3988432745 conn(0x7ff6700f1b10 msgr2=0x7ff6700f5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:56.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 shutdown_connections 2024-02-10T08:15:56.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:56.096+0000 7ff675744700 1 -- 172.21.15.134:0/3988432745 wait complete. 2024-02-10T08:15:56.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:15:56 smithi067 bash[17647]: Warning: Permanently added '172.21.15.134' (ECDSA) to the list of known hosts. 2024-02-10T08:15:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:56 smithi067 bash[17385]: cluster 2024-02-10T08:15:54.816561+0000 mgr.y (mgr.14146) 33 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:56.806 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T08:15:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:57 smithi067 bash[17385]: audit 2024-02-10T08:15:55.260074+0000 mgr.y (mgr.14146) 34 : audit [DBG] from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:15:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:57 smithi067 bash[17385]: audit 2024-02-10T08:15:56.094234+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:57 smithi067 bash[17385]: audit 2024-02-10T08:15:56.109018+0000 mon.a (mon.0) 104 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:15:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:57 smithi067 bash[17385]: audit 2024-02-10T08:15:56.651588+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:58 smithi067 bash[17385]: cephadm 2024-02-10T08:15:56.095053+0000 mgr.y (mgr.14146) 35 : cephadm [INF] Added host smithi134 2024-02-10T08:15:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:58 smithi067 bash[17385]: cluster 2024-02-10T08:15:56.817071+0000 mgr.y (mgr.14146) 36 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:15:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:15:58 smithi067 bash[17385]: audit 2024-02-10T08:15:57.897959+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:15:59.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- 172.21.15.134:0/2344867529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 msgr2=0x7f40f80f69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:59.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 --2- 172.21.15.134:0/2344867529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f80f69a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f40e8009ae0 tx=0x7f40e8004d10).stop 2024-02-10T08:15:59.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- 172.21.15.134:0/2344867529 shutdown_connections 2024-02-10T08:15:59.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 --2- 172.21.15.134:0/2344867529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f80f69a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:59.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- 172.21.15.134:0/2344867529 >> 172.21.15.134:0/2344867529 conn(0x7f40f80f1ce0 msgr2=0x7f40f80f4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:59.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- 172.21.15.134:0/2344867529 shutdown_connections 2024-02-10T08:15:59.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- 172.21.15.134:0/2344867529 wait complete. 2024-02-10T08:15:59.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 Processor -- start 2024-02-10T08:15:59.265 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- start start 2024-02-10T08:15:59.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f8105e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:59.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40fdccd700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f8106310 con 0x7f40f80f6580 2024-02-10T08:15:59.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40f77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f8105e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:59.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40f77fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f8105e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.134:34138/0 (socket says 172.21.15.134:34138) 2024-02-10T08:15:59.266 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.260+0000 7f40f77fe700 1 -- 172.21.15.134:0/426625977 learned_addr learned my addr 172.21.15.134:0/426625977 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:15:59.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f77fe700 1 -- 172.21.15.134:0/426625977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80097c0 con 0x7f40f80f6580 2024-02-10T08:15:59.267 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f77fe700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f8105e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f40e8007ec0 tx=0x7f40e8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:59.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40e800e030 con 0x7f40f80f6580 2024-02-10T08:15:59.268 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f40e800bc30 con 0x7f40f80f6580 2024-02-10T08:15:59.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40e8018070 con 0x7f40f80f6580 2024-02-10T08:15:59.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f8106510 con 0x7f40f80f6580 2024-02-10T08:15:59.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f8107120 con 0x7f40f80f6580 2024-02-10T08:15:59.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f81069a0 con 0x7f40f80f6580 2024-02-10T08:15:59.269 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f40e801d420 con 0x7f40f80f6580 2024-02-10T08:15:59.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f4ff9700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f40e0032700 0x7f40e0034bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:15:59.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f40e8044660 con 0x7f40f80f6580 2024-02-10T08:15:59.270 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.264+0000 7f40f6ffd700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f40e0032700 0x7f40e0034bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:15:59.271 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.268+0000 7f40f6ffd700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f40e0032700 0x7f40e0034bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f40ec009780 tx=0x7f40ec006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:15:59.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.276+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f40f81069a0 con 0x7f40f80f6580 2024-02-10T08:15:59.473 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.468+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40f80fb830 con 0x7f40e0032700 2024-02-10T08:15:59.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40f4ff9700 1 -- 172.21.15.134:0/426625977 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f40f80fb830 con 0x7f40e0032700 2024-02-10T08:15:59.477 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-02-10T08:15:59.477 INFO:teuthology.orchestra.run.smithi134.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f40e0032700 msgr2=0x7f40e0034bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f40e0032700 0x7f40e0034bc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f40ec009780 tx=0x7f40ec006cc0).stop 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 msgr2=0x7f40f8105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f8105e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f40e8007ec0 tx=0x7f40e8004d10).stop 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 shutdown_connections 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f40e0032700 0x7f40e0034bc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:59.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.472+0000 7f40fdccd700 1 --2- 172.21.15.134:0/426625977 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f40f80f6580 0x7f40f8105e20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:15:59.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.476+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 >> 172.21.15.134:0/426625977 conn(0x7f40f80f1ce0 msgr2=0x7f40f80f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:15:59.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.476+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 shutdown_connections 2024-02-10T08:15:59.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:15:59.476+0000 7f40fdccd700 1 -- 172.21.15.134:0/426625977 wait complete. 2024-02-10T08:16:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:00 smithi067 bash[17385]: cluster 2024-02-10T08:15:58.817485+0000 mgr.y (mgr.14146) 37 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:00.402 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-10T08:16:00.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd crush tunables default 2024-02-10T08:16:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:01 smithi067 bash[17385]: audit 2024-02-10T08:15:59.475369+0000 mgr.y (mgr.14146) 38 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:16:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:02 smithi067 bash[17385]: cluster 2024-02-10T08:16:00.817861+0000 mgr.y (mgr.14146) 39 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:03.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:03 smithi067 bash[17385]: cluster 2024-02-10T08:16:02.818304+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 -- 172.21.15.67:0/3129275589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 msgr2=0x7f91c80fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 --2- 172.21.15.67:0/3129275589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c80fdaa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f91b8004770 tx=0x7f91b8009cb0).stop 2024-02-10T08:16:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 -- 172.21.15.67:0/3129275589 shutdown_connections 2024-02-10T08:16:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 --2- 172.21.15.67:0/3129275589 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c80fdaa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 -- 172.21.15.67:0/3129275589 >> 172.21.15.67:0/3129275589 conn(0x7f91c80f32c0 msgr2=0x7f91c80f56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:03.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 -- 172.21.15.67:0/3129275589 shutdown_connections 2024-02-10T08:16:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 -- 172.21.15.67:0/3129275589 wait complete. 2024-02-10T08:16:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 Processor -- start 2024-02-10T08:16:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.505+0000 7f91cd026700 1 -- start start 2024-02-10T08:16:03.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91cd026700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c810ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91cd026700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c810d3a0 con 0x7f91c80fd6c0 2024-02-10T08:16:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c810ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c810ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50114/0 (socket says 172.21.15.67:50114) 2024-02-10T08:16:03.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c6d9d700 1 -- 172.21.15.67:0/1232037612 learned_addr learned my addr 172.21.15.67:0/1232037612 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:16:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c6d9d700 1 -- 172.21.15.67:0/1232037612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8005040 con 0x7f91c80fd6c0 2024-02-10T08:16:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c6d9d700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c810ceb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f91b8009770 tx=0x7f91b8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91b800e030 con 0x7f91c80fd6c0 2024-02-10T08:16:03.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f91b8013dc0 con 0x7f91c80fd6c0 2024-02-10T08:16:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c810d5a0 con 0x7f91c80fd6c0 2024-02-10T08:16:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91b8018070 con 0x7f91c80fd6c0 2024-02-10T08:16:03.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c810da30 con 0x7f91c80fd6c0 2024-02-10T08:16:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.509+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f91b801c3c0 con 0x7f91c80fd6c0 2024-02-10T08:16:03.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.513+0000 7f91c4d14700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f91b40323c0 0x7f91b4034880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.513+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f91b8049630 con 0x7f91c80fd6c0 2024-02-10T08:16:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.513+0000 7f91bffff700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f91b40323c0 0x7f91b4034880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.513+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c805f1f0 con 0x7f91c80fd6c0 2024-02-10T08:16:03.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.513+0000 7f91bffff700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f91b40323c0 0x7f91b4034880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f91b0009780 tx=0x7f91b0006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:03.524 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.521+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f91b804a060 con 0x7f91c80fd6c0 2024-02-10T08:16:03.715 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:03.713+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f91c805a3b0 con 0x7f91c80fd6c0 2024-02-10T08:16:04.724 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.721+0000 7f91c4d14700 1 -- 172.21.15.67:0/1232037612 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f91b801aa80 con 0x7f91c80fd6c0 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f91b40323c0 msgr2=0x7f91b4034880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f91b40323c0 0x7f91b4034880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f91b0009780 tx=0x7f91b0006cc0).stop 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 msgr2=0x7f91c810ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c810ceb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f91b8009770 tx=0x7f91b8009cb0).stop 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 shutdown_connections 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f91b40323c0 0x7f91b4034880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 --2- 172.21.15.67:0/1232037612 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f91c80fd6c0 0x7f91c810ceb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 >> 172.21.15.67:0/1232037612 conn(0x7f91c80f32c0 msgr2=0x7f91c80f3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 shutdown_connections 2024-02-10T08:16:04.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:04.725+0000 7f91cd026700 1 -- 172.21.15.67:0/1232037612 wait complete. 2024-02-10T08:16:04.729 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-02-10T08:16:04.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:03.718179+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:04.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:03.718480+0000 mon.a (mon.0) 108 : audit [INF] from='client.? 172.21.15.67:0/1232037612' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T08:16:04.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:03.999934+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:04.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:04.001473+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:16:04.988 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:04.003734+0000 mon.a (mon.0) 111 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:04.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:04.005144+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:04.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: cephadm 2024-02-10T08:16:04.007030+0000 mgr.y (mgr.14146) 41 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:16:04.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: cephadm 2024-02-10T08:16:04.029849+0000 mgr.y (mgr.14146) 42 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:16:04.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:04.062400+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:04.989 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:04 smithi067 bash[17385]: audit 2024-02-10T08:16:04.066892+0000 mon.a (mon.0) 114 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:05.509 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-02-10T08:16:05.509 INFO:tasks.cephadm:Adding mon.c on smithi067 2024-02-10T08:16:05.509 INFO:tasks.cephadm:Adding mon.b on smithi134 2024-02-10T08:16:05.510 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi067:[v2:172.21.15.67:3301,v1:172.21.15.67:6790]=c;smithi134:172.21.15.134=b' 2024-02-10T08:16:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:05 smithi067 bash[17385]: audit 2024-02-10T08:16:04.724346+0000 mon.a (mon.0) 115 : audit [INF] from='client.? 172.21.15.67:0/1232037612' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T08:16:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:05 smithi067 bash[17385]: cluster 2024-02-10T08:16:04.724437+0000 mon.a (mon.0) 116 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:05 smithi067 bash[17385]: cluster 2024-02-10T08:16:04.818722+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:08.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.000+0000 7fd2aa989700 1 -- 172.21.15.134:0/898425344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 msgr2=0x7fd2a40fc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:08.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.000+0000 7fd2aa989700 1 --2- 172.21.15.134:0/898425344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a40fc980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7fd294004770 tx=0x7fd294009cb0).stop 2024-02-10T08:16:08.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.000+0000 7fd2aa989700 1 -- 172.21.15.134:0/898425344 shutdown_connections 2024-02-10T08:16:08.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.000+0000 7fd2aa989700 1 --2- 172.21.15.134:0/898425344 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a40fc980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:08.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.000+0000 7fd2aa989700 1 -- 172.21.15.134:0/898425344 >> 172.21.15.134:0/898425344 conn(0x7fd2a40f21a0 msgr2=0x7fd2a40f45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:08.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.000+0000 7fd2aa989700 1 -- 172.21.15.134:0/898425344 shutdown_connections 2024-02-10T08:16:08.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 -- 172.21.15.134:0/898425344 wait complete. 2024-02-10T08:16:08.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 Processor -- start 2024-02-10T08:16:08.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 -- start start 2024-02-10T08:16:08.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a410f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:08.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a410f5a0 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a410f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:08.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a3fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a410f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.134:34156/0 (socket says 172.21.15.134:34156) 2024-02-10T08:16:08.009 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a3fff700 1 -- 172.21.15.134:0/21674378 learned_addr learned my addr 172.21.15.134:0/21674378 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:16:08.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a3fff700 1 -- 172.21.15.134:0/21674378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd294005040 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a3fff700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a410f0b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd294009770 tx=0x7fd294009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:08.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd29400e030 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a410f7a0 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fd294013dc0 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.010 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd294018070 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.004+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a410fc30 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.008+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a405fee0 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.011 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.008+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fd294027440 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.008+0000 7fd2a17fa700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd2900326d0 0x7fd290034b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:08.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.008+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fd2940440f0 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.008+0000 7fd2a37fe700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd2900326d0 0x7fd290034b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:08.013 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.008+0000 7fd2a37fe700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd2900326d0 0x7fd290034b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fd28c009780 tx=0x7fd28c006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:08.021 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.016+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fd29401b960 con 0x7fd2a40fc5a0 2024-02-10T08:16:08.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.212+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi067:[v2:172.21.15.67:3301,v1:172.21.15.67:6790]=c;smithi134:172.21.15.134=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2a410a730 con 0x7fd2900326d0 2024-02-10T08:16:08.232 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-02-10T08:16:08.232 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.228+0000 7fd2a17fa700 1 -- 172.21.15.134:0/21674378 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd2a410a730 con 0x7fd2900326d0 2024-02-10T08:16:08.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd2900326d0 msgr2=0x7fd290034b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:08.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd2900326d0 0x7fd290034b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fd28c009780 tx=0x7fd28c006cc0).stop 2024-02-10T08:16:08.235 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 msgr2=0x7fd2a410f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a410f0b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fd294009770 tx=0x7fd294009cb0).stop 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 shutdown_connections 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fd2900326d0 0x7fd290034b90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 --2- 172.21.15.134:0/21674378 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fd2a40fc5a0 0x7fd2a410f0b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 >> 172.21.15.134:0/21674378 conn(0x7fd2a40f21a0 msgr2=0x7fd2a4060f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 shutdown_connections 2024-02-10T08:16:08.236 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:08.232+0000 7fd2aa989700 1 -- 172.21.15.134:0/21674378 wait complete. 2024-02-10T08:16:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:07 smithi067 bash[17385]: cluster 2024-02-10T08:16:06.819213+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:09.092 DEBUG:teuthology.orchestra.run.smithi067:mon.c> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.c.service 2024-02-10T08:16:09.094 DEBUG:teuthology.orchestra.run.smithi134:mon.b> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.b.service 2024-02-10T08:16:09.097 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T08:16:09.097 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T08:16:09.308 INFO:journalctl@ceph.mon.c.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:16:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.220096+0000 mgr.y (mgr.14146) 45 : audit [DBG] from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi067:[v2:172.21.15.67:3301,v1:172.21.15.67:6790]=c;smithi134:172.21.15.134=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: cephadm 2024-02-10T08:16:08.226210+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Saving service mon spec with placement smithi067:172.21.15.67=a;smithi067:[v2:172.21.15.67:3301,v1:172.21.15.67:6790]=c;smithi134:172.21.15.134=b;count:3 2024-02-10T08:16:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.231061+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.242563+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.539131+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.540610+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.546254+0000 mon.a (mon.0) 121 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:09.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.549025+0000 mon.a (mon.0) 122 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:09.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: audit 2024-02-10T08:16:08.550129+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:09.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: cephadm 2024-02-10T08:16:08.551088+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Deploying daemon mon.b on smithi134 2024-02-10T08:16:09.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:09 smithi067 bash[17385]: cluster 2024-02-10T08:16:08.819560+0000 mgr.y (mgr.14146) 48 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:16:12.369 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:11 smithi067 bash[17385]: cluster 2024-02-10T08:16:10.819902+0000 mgr.y (mgr.14146) 49 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:13.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 -- 172.21.15.134:0/4002980794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 msgr2=0x7fe3680fc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:13.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 --2- 172.21.15.134:0/4002980794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe3680fc9d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe358004770 tx=0x7fe358009cb0).stop 2024-02-10T08:16:13.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 -- 172.21.15.134:0/4002980794 shutdown_connections 2024-02-10T08:16:13.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 --2- 172.21.15.134:0/4002980794 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe3680fc9d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:13.250 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 -- 172.21.15.134:0/4002980794 >> 172.21.15.134:0/4002980794 conn(0x7fe36806c4b0 msgr2=0x7fe36806c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:13.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 -- 172.21.15.134:0/4002980794 shutdown_connections 2024-02-10T08:16:13.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.244+0000 7fe36e27c700 1 -- 172.21.15.134:0/4002980794 wait complete. 2024-02-10T08:16:13.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 Processor -- start 2024-02-10T08:16:13.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 -- start start 2024-02-10T08:16:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe36810cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36810d490 con 0x7fe3680fc5f0 2024-02-10T08:16:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe367fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe36810cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe367fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe36810cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.134:43900/0 (socket says 172.21.15.134:43900) 2024-02-10T08:16:13.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe367fff700 1 -- 172.21.15.134:0/2795299315 learned_addr learned my addr 172.21.15.134:0/2795299315 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:16:13.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe367fff700 1 -- 172.21.15.134:0/2795299315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358005040 con 0x7fe3680fc5f0 2024-02-10T08:16:13.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe367fff700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe36810cfa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe358009770 tx=0x7fe358009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:13.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe35800e030 con 0x7fe3680fc5f0 2024-02-10T08:16:13.253 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36810d690 con 0x7fe3680fc5f0 2024-02-10T08:16:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fe358013dc0 con 0x7fe3680fc5f0 2024-02-10T08:16:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe358018070 con 0x7fe3680fc5f0 2024-02-10T08:16:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36810daa0 con 0x7fe3680fc5f0 2024-02-10T08:16:13.254 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.248+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36805fee0 con 0x7fe3680fc5f0 2024-02-10T08:16:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.252+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fe35801d520 con 0x7fe3680fc5f0 2024-02-10T08:16:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.252+0000 7fe365775700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fe350032720 0x7fe350034be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:13.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.252+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fe358044080 con 0x7fe3680fc5f0 2024-02-10T08:16:13.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.252+0000 7fe3677fe700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fe350032720 0x7fe350034be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:13.256 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.252+0000 7fe3677fe700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fe350032720 0x7fe350034be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fe35c009780 tx=0x7fe35c006cc0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:13.264 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.260+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe358025990 con 0x7fe3680fc5f0 2024-02-10T08:16:13.518 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.512+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe368046d80 con 0x7fe3680fc5f0 2024-02-10T08:16:13.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe365775700 1 -- 172.21.15.134:0/2795299315 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+723 (secure 0 0 0) 0x7fe358012e80 con 0x7fe3680fc5f0 2024-02-10T08:16:13.519 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-02-10T08:16:13.519 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"2dd26d0c-c7ec-11ee-95b8-87774f69a715","modified":"2024-02-10T08:13:41.042140Z","created":"2024-02-10T08:13:41.042140Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fe350032720 msgr2=0x7fe350034be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fe350032720 0x7fe350034be0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fe35c009780 tx=0x7fe35c006cc0).stop 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 msgr2=0x7fe36810cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe36810cfa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fe358009770 tx=0x7fe358009cb0).stop 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 shutdown_connections 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fe350032720 0x7fe350034be0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 --2- 172.21.15.134:0/2795299315 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe3680fc5f0 0x7fe36810cfa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 >> 172.21.15.134:0/2795299315 conn(0x7fe36806c4b0 msgr2=0x7fe368060630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 shutdown_connections 2024-02-10T08:16:13.522 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:13.516+0000 7fe36e27c700 1 -- 172.21.15.134:0/2795299315 wait complete. 2024-02-10T08:16:13.523 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-02-10T08:16:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:13 smithi067 bash[17385]: cluster 2024-02-10T08:16:12.820234+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:13 smithi067 bash[17385]: audit 2024-02-10T08:16:13.519277+0000 mon.a (mon.0) 124 : audit [DBG] from='client.? 172.21.15.134:0/2795299315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T08:16:14.564 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:14 smithi134 systemd[1]: Started Ceph mon.b for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:16:15.072 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T08:16:15.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T08:16:15.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:15 smithi067 bash[17385]: audit 2024-02-10T08:16:14.537065+0000 mon.a (mon.0) 125 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:15.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:15 smithi067 bash[17385]: audit 2024-02-10T08:16:14.538899+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:15.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:15 smithi067 bash[17385]: audit 2024-02-10T08:16:14.540368+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:15.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:15 smithi067 bash[17385]: cephadm 2024-02-10T08:16:14.541687+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Deploying daemon mon.c on smithi067 2024-02-10T08:16:15.863 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:15 smithi067 bash[17385]: cluster 2024-02-10T08:16:14.820599+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.660+0000 7f325cc5d880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.660+0000 7f325cc5d880 0 ceph version 16.2.14-521-g0e714d9a (0e714d9a4bd2a821113e6318adb87bd06cf81ec1) pacific (stable), process ceph-mon, pid 7 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.660+0000 7f325cc5d880 0 pidfile_write: ignore empty --pid-file 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 0 load: jerasure load: lrc load: isa 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: RocksDB version: 6.8.1 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-10T08:16:16.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Compile date Feb 8 2024 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: DB SUMMARY 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: CURRENT file: CURRENT 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 511 ; 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.error_if_exists: 0 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.create_if_missing: 0 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.paranoid_checks: 1 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.env: 0x55923a426080 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.fs: Posix File System 2024-02-10T08:16:16.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.info_log: 0x55923bc66220 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.statistics: (nil) 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.use_fsync: 0 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_log_file_size: 0 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.allow_fallocate: 1 2024-02-10T08:16:16.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.use_direct_reads: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.db_log_dir: 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_subcompactions: 1 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_background_flushes: -1 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T08:16:16.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.write_buffer_manager: 0x55923c95a3f0 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-10T08:16:16.032 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-10T08:16:16.034 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-10T08:16:16.034 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.unordered_write: 0 2024-02-10T08:16:16.034 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T08:16:16.034 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T08:16:16.034 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.row_cache: None 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.wal_filter: None 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.preserve_deletes: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.two_write_queues: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.atomic_flush: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T08:16:16.035 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.log_readahead_size: 0 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_background_jobs: 2 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_background_compactions: -1 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T08:16:16.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_open_files: -1 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Compression algorithms supported: 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kZSTD supported: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kXpressCompression supported: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kLZ4Compression supported: 1 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kBZip2Compression supported: 0 2024-02-10T08:16:16.037 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kZlibCompression supported: 1 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: kSnappyCompression supported: 1 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.merge_operator: 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_filter: None 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55923bc09d28) 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: cache_index_and_filter_blocks: 1 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T08:16:16.038 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: pin_top_level_index_and_filter: 1 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: index_type: 0 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: data_block_index_type: 0 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: index_shortening: 1 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: data_block_hash_table_util_ratio: 0.750000 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: hash_index_allow_collision: 1 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: checksum: 1 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: no_block_cache: 0 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_cache: 0x55923bc40f10 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_cache_name: BinnedLRUCache 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_cache_options: 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: capacity : 536870912 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: num_shard_bits : 4 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: strict_capacity_limit : 0 2024-02-10T08:16:16.039 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: high_pri_pool_ratio: 0.000 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_cache_compressed: (nil) 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: persistent_cache: (nil) 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_size: 4096 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_size_deviation: 10 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_restart_interval: 16 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: index_block_restart_interval: 1 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: metadata_block_size: 4096 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: partition_filters: 0 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: use_delta_encoding: 1 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: whole_key_filtering: 1 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: verify_compression: 0 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: read_amp_bytes_per_bit: 0 2024-02-10T08:16:16.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: format_version: 2 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: enable_index_compression: 1 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: block_align: 0 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression: NoCompression 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.num_levels: 7 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T08:16:16.041 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-10T08:16:16.042 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-10T08:16:16.043 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.table_properties_collectors: 2024-02-10T08:16:16.044 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.inplace_update_support: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.bloom_locality: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.max_successive_merges: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.ttl: 2592000 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707552975674503, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-10T08:16:16.045 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.668+0000 7f325cc5d880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707552975675790, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1437, "table_properties": {"data_size": 523, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1707552975, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1707552975676514, "job": 1, "event": "recovery_finished"} 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 4 rocksdb: DB pointer 0x55923bd09800 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f3243a4e700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f3243a4e700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** DB Stats ** 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-10T08:16:16.046 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** Compaction Stats [default] ** 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** Compaction Stats [default] ** 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T08:16:16.047 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(Keys): cumulative 0, interval 0 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Interval compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** File Read Latency Histogram By Level [default] ** 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** Compaction Stats [default] ** 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** Compaction Stats [default] ** 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-10T08:16:16.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 0.00 0.00 1 0.001 0 0 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: AddFile(Keys): cumulative 0, interval 0 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: ** File Read Latency Histogram By Level [default] ** 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.672+0000 7f325cc5d880 0 using public_addr v2:172.21.15.134:0/0 -> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.676+0000 7f325cc5d880 0 starting mon.b rank -1 at public addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] at bind addrs [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:16:16.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.676+0000 7f325cc5d880 1 mon.b@-1(???) e0 preinit fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 0 mon.b@-1(synchronizing).mds e1 new map 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: e1 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: legacy client fscid: -1 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: No filesystems configured 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-10T08:16:16.050 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: audit 2024-02-10T08:16:14.537065+0000 mon.a (mon.0) 125 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: audit 2024-02-10T08:16:14.538899+0000 mon.a (mon.0) 126 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: audit 2024-02-10T08:16:14.540368+0000 mon.a (mon.0) 127 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: cephadm 2024-02-10T08:16:14.541687+0000 mgr.y (mgr.14146) 51 : cephadm [INF] Deploying daemon mon.c on smithi067 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: cluster 2024-02-10T08:16:14.820599+0000 mgr.y (mgr.14146) 52 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:16.051 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:15 smithi134 bash[18722]: debug 2024-02-10T08:16:15.704+0000 7f3246253700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T08:16:19.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:19 smithi067 bash[24488]: debug 2024-02-10T08:16:19.705+0000 7f0f40d76700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T08:16:20.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 -- 172.21.15.134:0/4218689119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0518003400 msgr2=0x7f0518005890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:20.814 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 --2- 172.21.15.134:0/4218689119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0518003400 0x7f0518005890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f052c06b060 tx=0x7f0528009040).stop 2024-02-10T08:16:20.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 -- 172.21.15.134:0/4218689119 shutdown_connections 2024-02-10T08:16:20.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 --2- 172.21.15.134:0/4218689119 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f0518003400 0x7f0518005890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:20.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 -- 172.21.15.134:0/4218689119 >> 172.21.15.134:0/4218689119 conn(0x7f052c06c4e0 msgr2=0x7f052c06c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:20.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 -- 172.21.15.134:0/4218689119 shutdown_connections 2024-02-10T08:16:20.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.807+0000 7f05334cb700 1 -- 172.21.15.134:0/4218689119 wait complete. 2024-02-10T08:16:20.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 Processor -- start 2024-02-10T08:16:20.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 -- start start 2024-02-10T08:16:20.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:20.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f052c10da70 0x7f052c107130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:20.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c107730 con 0x7f052c10da70 2024-02-10T08:16:20.816 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c107870 con 0x7f052c10d1a0 2024-02-10T08:16:20.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:20.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46216/0 (socket says 172.21.15.134:46216) 2024-02-10T08:16:20.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 -- 172.21.15.134:0/4108116294 learned_addr learned my addr 172.21.15.134:0/4108116294 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:16:20.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0530a66700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f052c10da70 0x7f052c107130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:20.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 -- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 msgr2=0x7f052c10d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-10T08:16:20.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 -- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 msgr2=0x7f052c10d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-10T08:16:20.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-10T08:16:20.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0531267700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:16:20.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0530a66700 1 -- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 msgr2=0x7f052c10d580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0530a66700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0530a66700 1 -- 172.21.15.134:0/4108116294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528007860 con 0x7f052c10da70 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f0530a66700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f052c10da70 0x7f052c107130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f051c00b460 tx=0x7f051c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f051c00f6c0 con 0x7f052c10da70 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c107b50 con 0x7f052c10da70 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f051c00fcb0 con 0x7f052c10da70 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f051c011ac0 con 0x7f052c10da70 2024-02-10T08:16:20.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.811+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c111fc0 con 0x7f052c10da70 2024-02-10T08:16:20.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.815+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f051c01a400 con 0x7f052c10da70 2024-02-10T08:16:20.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.815+0000 7f05227fc700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f0510032840 0x7f0510034d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:20.820 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.815+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f051c043b50 con 0x7f052c10da70 2024-02-10T08:16:20.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.815+0000 7f0531267700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f0510032840 0x7f0510034d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:20.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.815+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c05f290 con 0x7f052c10da70 2024-02-10T08:16:20.821 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.815+0000 7f0531267700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f0510032840 0x7f0510034d00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f0528013680 tx=0x7f0528009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:20.831 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:20.823+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f051c011d80 con 0x7f052c10da70 2024-02-10T08:16:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:15.719350+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:16:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:15.719757+0000 mon.a (mon.0) 130 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:15.744361+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:16.715728+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:16.821025+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:17.716019+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:17.716949+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:18.715293+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:18.821455+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:19.712432+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:19.715484+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:20.712689+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:20.716037+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:20.765477+0000 mon.a (mon.0) 139 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:20.770894+0000 mon.a (mon.0) 140 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2024-02-10T08:16:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:20.791662+0000 mon.a (mon.0) 141 : cluster [DBG] fsmap 2024-02-10T08:16:21.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:20.791756+0000 mon.a (mon.0) 142 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:21.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:20.792185+0000 mon.a (mon.0) 143 : cluster [DBG] mgrmap e13: y(active, since 95s) 2024-02-10T08:16:21.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: cluster 2024-02-10T08:16:20.800731+0000 mon.a (mon.0) 144 : cluster [INF] overall HEALTH_OK 2024-02-10T08:16:21.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:20.806564+0000 mon.a (mon.0) 145 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:21.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:20 smithi067 bash[17385]: audit 2024-02-10T08:16:20.808394+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:21.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.143+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f052c05a3b0 con 0x7f052c10da70 2024-02-10T08:16:21.150 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.143+0000 7f05227fc700 1 -- 172.21.15.134:0/4108116294 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+993 (secure 0 0 0) 0x7f051c00e260 con 0x7f052c10da70 2024-02-10T08:16:21.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-02-10T08:16:21.150 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"2dd26d0c-c7ec-11ee-95b8-87774f69a715","modified":"2024-02-10T08:16:15.715029Z","created":"2024-02-10T08:13:41.042140Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f0510032840 msgr2=0x7f0510034d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f0510032840 0x7f0510034d00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f0528013680 tx=0x7f0528009040).stop 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f052c10da70 msgr2=0x7f052c107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f052c10da70 0x7f052c107130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f051c00b460 tx=0x7f051c009060).stop 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 shutdown_connections 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f0510032840 0x7f0510034d00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f052c10d1a0 0x7f052c10d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 --2- 172.21.15.134:0/4108116294 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f052c10da70 0x7f052c107130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 >> 172.21.15.134:0/4108116294 conn(0x7f052c06c4e0 msgr2=0x7f052c060e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 shutdown_connections 2024-02-10T08:16:21.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:21.147+0000 7f05334cb700 1 -- 172.21.15.134:0/4108116294 wait complete. 2024-02-10T08:16:21.155 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:15.719350+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:15.719757+0000 mon.a (mon.0) 130 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:15.744361+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:16.715728+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:16.821025+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:17.716019+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:17.716949+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:18.715293+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:18.821455+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:19.712432+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:21.203 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:19.715484+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:20.712689+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:20.716037+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:20.765477+0000 mon.a (mon.0) 139 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:20.770894+0000 mon.a (mon.0) 140 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:20.791662+0000 mon.a (mon.0) 141 : cluster [DBG] fsmap 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:20.791756+0000 mon.a (mon.0) 142 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:20.792185+0000 mon.a (mon.0) 143 : cluster [DBG] mgrmap e13: y(active, since 95s) 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: cluster 2024-02-10T08:16:20.800731+0000 mon.a (mon.0) 144 : cluster [INF] overall HEALTH_OK 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:20.806564+0000 mon.a (mon.0) 145 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:21.204 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:20 smithi134 bash[18722]: audit 2024-02-10T08:16:20.808394+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:22.849 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T08:16:22.849 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T08:16:23.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:16:22 smithi067 bash[17647]: debug 2024-02-10T08:16:22.713+0000 7f9600d7d700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.791+0000 7f56c3385700 1 -- 172.21.15.134:0/3969853909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc103c70 msgr2=0x7f56bc104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.791+0000 7f56c3385700 1 --2- 172.21.15.134:0/3969853909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc103c70 0x7f56bc104050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f56b8009fd0 tx=0x7f56b800b040).stop 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.791+0000 7f56c3385700 1 -- 172.21.15.134:0/3969853909 shutdown_connections 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.791+0000 7f56c3385700 1 --2- 172.21.15.134:0/3969853909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc103c70 0x7f56bc104050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- 172.21.15.134:0/3969853909 >> 172.21.15.134:0/3969853909 conn(0x7f56bc06d620 msgr2=0x7f56bc06fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- 172.21.15.134:0/3969853909 shutdown_connections 2024-02-10T08:16:26.800 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- 172.21.15.134:0/3969853909 wait complete. 2024-02-10T08:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 Processor -- start 2024-02-10T08:16:26.801 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- start start 2024-02-10T08:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 0x7f56bc10d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:26.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc10dc90 0x7f56bc10e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc114320 con 0x7f56bc10dc90 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc114460 con 0x7f56bc103c70 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c3385700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc11a640 con 0x7f56bc113d00 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46232/0 (socket says 172.21.15.134:46232) 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 -- 172.21.15.134:0/1477296664 learned_addr learned my addr 172.21.15.134:0/1477296664 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc10dc90 0x7f56bc10e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c0920700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 0x7f56bc10d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 msgr2=0x7f56bc113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 msgr2=0x7f56bc113810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-10T08:16:26.803 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1121700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c0920700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 msgr2=0x7f56bc10d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c0920700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 msgr2=0x7f56bc10d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c0920700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 0x7f56bc10d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c0920700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 0x7f56bc10d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 msgr2=0x7f56bc10d7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 0x7f56bc10d7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 msgr2=0x7f56bc113810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:26.804 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 -- 172.21.15.134:0/1477296664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8005040 con 0x7f56bc10dc90 2024-02-10T08:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.795+0000 7f56c1922700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc10dc90 0x7f56bc10e110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f56b400bb00 tx=0x7f56b40096b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b400c8f0 con 0x7f56bc10dc90 2024-02-10T08:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc11a920 con 0x7f56bc10dc90 2024-02-10T08:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f56b4015420 con 0x7f56bc10dc90 2024-02-10T08:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b40143d0 con 0x7f56bc10dc90 2024-02-10T08:16:26.805 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc11ae90 con 0x7f56bc10dc90 2024-02-10T08:16:26.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc05f100 con 0x7f56bc10dc90 2024-02-10T08:16:26.806 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f56b4012020 con 0x7f56bc10dc90 2024-02-10T08:16:26.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56b27fc700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f56a8032ba0 0x7f56a8035060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:26.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.799+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f56b4043050 con 0x7f56bc10dc90 2024-02-10T08:16:26.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.803+0000 7f56c1121700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f56a8032ba0 0x7f56a8035060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:26.807 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.803+0000 7f56c1121700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f56a8032ba0 0x7f56a8035060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f56b8009c00 tx=0x7f56b800b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:26.815 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:26.811+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f56b4009720 con 0x7f56bc10dc90 2024-02-10T08:16:27.073 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.067+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f56bc107280 con 0x7f56bc10dc90 2024-02-10T08:16:27.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.067+0000 7f56b27fc700 1 -- 172.21.15.134:0/1477296664 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1259 (secure 0 0 0) 0x7f56b4023500 con 0x7f56bc10dc90 2024-02-10T08:16:27.074 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-02-10T08:16:27.074 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"2dd26d0c-c7ec-11ee-95b8-87774f69a715","modified":"2024-02-10T08:16:21.713150Z","created":"2024-02-10T08:13:41.042140Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3301","nonce":0},{"type":"v1","addr":"172.21.15.67:6790","nonce":0}]},"addr":"172.21.15.67:6790/0","public_addr":"172.21.15.67:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-10T08:16:27.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f56a8032ba0 msgr2=0x7f56a8035060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:27.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f56a8032ba0 0x7f56a8035060 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f56b8009c00 tx=0x7f56b800b040).stop 2024-02-10T08:16:27.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc10dc90 msgr2=0x7f56bc10e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:27.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc10dc90 0x7f56bc10e110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f56b400bb00 tx=0x7f56b40096b0).stop 2024-02-10T08:16:27.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 shutdown_connections 2024-02-10T08:16:27.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f56a8032ba0 0x7f56a8035060 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f56bc10dc90 0x7f56bc10e110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56bc103c70 0x7f56bc113810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 --2- 172.21.15.134:0/1477296664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f56bc113d00 0x7f56bc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 >> 172.21.15.134:0/1477296664 conn(0x7f56bc06d620 msgr2=0x7f56bc06ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 shutdown_connections 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:27.071+0000 7f56c3385700 1 -- 172.21.15.134:0/1477296664 wait complete. 2024-02-10T08:16:27.077 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-02-10T08:16:27.078 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:15.719350+0000 mon.a (mon.0) 129 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:16:27.078 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:15.719757+0000 mon.a (mon.0) 130 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.078 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:15.744361+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.078 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:16.715728+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.078 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:16.821025+0000 mgr.y (mgr.14146) 53 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.078 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:17.716019+0000 mon.a (mon.0) 133 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:17.716949+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:18.715293+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:18.821455+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:19.712432+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:19.715484+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:20.712689+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:20.716037+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:20.765477+0000 mon.a (mon.0) 139 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:20.770894+0000 mon.a (mon.0) 140 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0]} removed_ranks: {} 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:20.791662+0000 mon.a (mon.0) 141 : cluster [DBG] fsmap 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:20.791756+0000 mon.a (mon.0) 142 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:20.792185+0000 mon.a (mon.0) 143 : cluster [DBG] mgrmap e13: y(active, since 95s) 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:20.800731+0000 mon.a (mon.0) 144 : cluster [INF] overall HEALTH_OK 2024-02-10T08:16:27.079 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:20.806564+0000 mon.a (mon.0) 145 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:20.808394+0000 mon.a (mon.0) 146 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:21.719224+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:21.719589+0000 mon.a (mon.0) 150 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:21.720089+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:21.740351+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:21.740667+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.080 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:22.712860+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:22.822272+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:23.713309+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:23.715049+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:24.713344+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:24.822690+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:25.713417+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:26.713584+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.723014+0000 mon.a (mon.0) 158 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.761238+0000 mon.a (mon.0) 159 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.770757+0000 mon.a (mon.0) 160 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0]} removed_ranks: {} 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.791643+0000 mon.a (mon.0) 161 : cluster [DBG] fsmap 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.791700+0000 mon.a (mon.0) 162 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:27.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.792051+0000 mon.a (mon.0) 163 : cluster [DBG] mgrmap e13: y(active, since 101s) 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.798684+0000 mon.a (mon.0) 164 : cluster [INF] overall HEALTH_OK 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:26.804158+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:26.808173+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:26.818420+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:26.819742+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[17385]: audit 2024-02-10T08:16:26.821069+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:21.719224+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:21.719589+0000 mon.a (mon.0) 150 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:21.720089+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:21.740351+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:21.740667+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:22.712860+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:22.822272+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:23.713309+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:23.715049+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:24.713344+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:24.822690+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:25.713417+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:26.713584+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.723014+0000 mon.a (mon.0) 158 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.761238+0000 mon.a (mon.0) 159 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.770757+0000 mon.a (mon.0) 160 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0]} removed_ranks: {} 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.791643+0000 mon.a (mon.0) 161 : cluster [DBG] fsmap 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.791700+0000 mon.a (mon.0) 162 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.792051+0000 mon.a (mon.0) 163 : cluster [DBG] mgrmap e13: y(active, since 101s) 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.798684+0000 mon.a (mon.0) 164 : cluster [INF] overall HEALTH_OK 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:26.804158+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.083 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:26.808173+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.084 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:26.818420+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.084 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:26.819742+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:27.084 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:26 smithi067 bash[24488]: audit 2024-02-10T08:16:26.821069+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:27.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:21.719224+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:16:27.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:21.719589+0000 mon.a (mon.0) 150 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:21.720089+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-10T08:16:27.122 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:21.740351+0000 mon.a (mon.0) 151 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:21.740667+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:22.712860+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:22.822272+0000 mgr.y (mgr.14146) 56 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:23.713309+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:23.715049+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:24.713344+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:24.822690+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:25.713417+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:26.713584+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.723014+0000 mon.a (mon.0) 158 : cluster [INF] mon.a calling monitor election 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.761238+0000 mon.a (mon.0) 159 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.770757+0000 mon.a (mon.0) 160 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0],c=[v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0]} removed_ranks: {} 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.791643+0000 mon.a (mon.0) 161 : cluster [DBG] fsmap 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.791700+0000 mon.a (mon.0) 162 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-10T08:16:27.123 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.792051+0000 mon.a (mon.0) 163 : cluster [DBG] mgrmap e13: y(active, since 101s) 2024-02-10T08:16:27.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.798684+0000 mon.a (mon.0) 164 : cluster [INF] overall HEALTH_OK 2024-02-10T08:16:27.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:26.804158+0000 mon.a (mon.0) 165 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:26.808173+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:26.818420+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:27.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:26.819742+0000 mon.a (mon.0) 168 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:27.124 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:26 smithi134 bash[18722]: audit 2024-02-10T08:16:26.821069+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:27.812 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-10T08:16:27.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph config generate-minimal-conf 2024-02-10T08:16:28.087 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: cephadm 2024-02-10T08:16:26.822736+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:16:28.087 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: cluster 2024-02-10T08:16:26.822991+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:28.087 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: cephadm 2024-02-10T08:16:26.823348+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:16:28.087 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:26.847117+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.087 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:26.854293+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:26.859975+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: cephadm 2024-02-10T08:16:26.861682+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:26.862001+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:26.862883+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:26.863777+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: cephadm 2024-02-10T08:16:26.864630+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Reconfiguring daemon mon.a on smithi067 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:27.073747+0000 mon.a (mon.0) 176 : audit [DBG] from='client.? 172.21.15.134:0/1477296664' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[24488]: audit 2024-02-10T08:16:27.713054+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: cephadm 2024-02-10T08:16:26.822736+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: cluster 2024-02-10T08:16:26.822991+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: cephadm 2024-02-10T08:16:26.823348+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:16:28.088 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:26.847117+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:26.854293+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:26.859975+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: cephadm 2024-02-10T08:16:26.861682+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:26.862001+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:26.862883+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:26.863777+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: cephadm 2024-02-10T08:16:26.864630+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Reconfiguring daemon mon.a on smithi067 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:27.073747+0000 mon.a (mon.0) 176 : audit [DBG] from='client.? 172.21.15.134:0/1477296664' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T08:16:28.089 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:27 smithi067 bash[17385]: audit 2024-02-10T08:16:27.713054+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:28.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: cephadm 2024-02-10T08:16:26.822736+0000 mgr.y (mgr.14146) 58 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:16:28.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: cluster 2024-02-10T08:16:26.822991+0000 mgr.y (mgr.14146) 59 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: cephadm 2024-02-10T08:16:26.823348+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:26.847117+0000 mon.a (mon.0) 170 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:26.854293+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:26.859975+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: cephadm 2024-02-10T08:16:26.861682+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:26.862001+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:26.862883+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:26.863777+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: cephadm 2024-02-10T08:16:26.864630+0000 mgr.y (mgr.14146) 62 : cephadm [INF] Reconfiguring daemon mon.a on smithi067 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:27.073747+0000 mon.a (mon.0) 176 : audit [DBG] from='client.? 172.21.15.134:0/1477296664' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T08:16:28.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:27 smithi134 bash[18722]: audit 2024-02-10T08:16:27.713054+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:16:30.153 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: cluster 2024-02-10T08:16:28.823385+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:30.153 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: audit 2024-02-10T08:16:29.555514+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:30.153 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: audit 2024-02-10T08:16:29.556987+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:30.153 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: audit 2024-02-10T08:16:29.558207+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:30.153 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: audit 2024-02-10T08:16:29.560138+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:30.154 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:29 smithi067 bash[17385]: cluster 2024-02-10T08:16:28.823385+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:30.154 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:29 smithi067 bash[17385]: audit 2024-02-10T08:16:29.555514+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:30.154 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:29 smithi067 bash[17385]: audit 2024-02-10T08:16:29.556987+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:30.154 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:29 smithi067 bash[17385]: audit 2024-02-10T08:16:29.558207+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:30.154 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:29 smithi067 bash[17385]: audit 2024-02-10T08:16:29.560138+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:29 smithi134 bash[18722]: cluster 2024-02-10T08:16:28.823385+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:29 smithi134 bash[18722]: audit 2024-02-10T08:16:29.555514+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:29 smithi134 bash[18722]: audit 2024-02-10T08:16:29.556987+0000 mon.a (mon.0) 179 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:29 smithi134 bash[18722]: audit 2024-02-10T08:16:29.558207+0000 mon.a (mon.0) 180 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:29 smithi134 bash[18722]: audit 2024-02-10T08:16:29.560138+0000 mon.a (mon.0) 181 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[17385]: cephadm 2024-02-10T08:16:29.556549+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-10T08:16:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[17385]: cephadm 2024-02-10T08:16:29.561448+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring daemon mon.c on smithi067 2024-02-10T08:16:31.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: cephadm 2024-02-10T08:16:29.556549+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-10T08:16:31.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:30 smithi067 bash[24488]: cephadm 2024-02-10T08:16:29.561448+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring daemon mon.c on smithi067 2024-02-10T08:16:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:30 smithi134 bash[18722]: cephadm 2024-02-10T08:16:29.556549+0000 mgr.y (mgr.14146) 64 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-10T08:16:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:30 smithi134 bash[18722]: cephadm 2024-02-10T08:16:29.561448+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring daemon mon.c on smithi067 2024-02-10T08:16:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:31 smithi067 bash[17385]: cluster 2024-02-10T08:16:30.823753+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:31 smithi067 bash[24488]: cluster 2024-02-10T08:16:30.823753+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:31 smithi134 bash[18722]: cluster 2024-02-10T08:16:30.823753+0000 mgr.y (mgr.14146) 66 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:32.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 -- 172.21.15.67:0/1538598156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f8c60 msgr2=0x7ffb580f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:32.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 --2- 172.21.15.67:0/1538598156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f8c60 0x7ffb580f9040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ffb48009c10 tx=0x7ffb48004d10).stop 2024-02-10T08:16:32.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 -- 172.21.15.67:0/1538598156 shutdown_connections 2024-02-10T08:16:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 --2- 172.21.15.67:0/1538598156 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f8c60 0x7ffb580f9040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 --2- 172.21.15.67:0/1538598156 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb58103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 --2- 172.21.15.67:0/1538598156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb580f9530 0x7ffb580f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 -- 172.21.15.67:0/1538598156 >> 172.21.15.67:0/1538598156 conn(0x7ffb5806dc60 msgr2=0x7ffb5806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.505+0000 7ffb60260700 1 -- 172.21.15.67:0/1538598156 shutdown_connections 2024-02-10T08:16:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- 172.21.15.67:0/1538598156 wait complete. 2024-02-10T08:16:32.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 Processor -- start 2024-02-10T08:16:32.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- start start 2024-02-10T08:16:32.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb580f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:32.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb580f8c60 0x7ffb580fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f9530 0x7ffb580fa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb581167a0 con 0x7ffb580f9530 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58116910 con 0x7ffb580f8c60 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58116ad0 con 0x7ffb580f75b0 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5dffc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb580f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f9530 0x7ffb580fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5dffc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb580f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:52450/0 (socket says 172.21.15.67:52450) 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5dffc700 1 -- 172.21.15.67:0/1272633384 learned_addr learned my addr 172.21.15.67:0/1272633384 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5d7fb700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb580f8c60 0x7ffb580fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 -- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 msgr2=0x7ffb580f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:32.514 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb580f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 -- 172.21.15.67:0/1272633384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb580f8c60 msgr2=0x7ffb580fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb580f8c60 0x7ffb580fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 -- 172.21.15.67:0/1272633384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50005040 con 0x7ffb580f9530 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5dffc700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb580f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb5e7fd700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f9530 0x7ffb580fa5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ffb500048f0 tx=0x7ffb50009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000e030 con 0x7ffb580f9530 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7ffb5000f040 con 0x7ffb580f9530 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb480097c0 con 0x7ffb580f9530 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500146a0 con 0x7ffb580f9530 2024-02-10T08:16:32.515 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb581171e0 con 0x7ffb580f9530 2024-02-10T08:16:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7ffb50014ad0 con 0x7ffb580f9530 2024-02-10T08:16:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb56ffd700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ffb440328f0 0x7ffb44034db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.509+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb58060070 con 0x7ffb580f9530 2024-02-10T08:16:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.513+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7ffb50042e70 con 0x7ffb580f9530 2024-02-10T08:16:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.513+0000 7ffb5dffc700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ffb440328f0 0x7ffb44034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.513+0000 7ffb5dffc700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ffb440328f0 0x7ffb44034db0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ffb480048c0 tx=0x7ffb48004d10).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:32.523 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.521+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ffb5001d3e0 con 0x7ffb580f9530 2024-02-10T08:16:32.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.705+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ffb5805f630 con 0x7ffb580f9530 2024-02-10T08:16:32.709 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.705+0000 7ffb56ffd700 1 -- 172.21.15.67:0/1272633384 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7ffb50012360 con 0x7ffb580f9530 2024-02-10T08:16:32.710 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:16:32.710 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-02-10T08:16:32.710 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:16:32.710 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ffb440328f0 msgr2=0x7ffb44034db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ffb440328f0 0x7ffb44034db0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ffb480048c0 tx=0x7ffb48004d10).stop 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f9530 msgr2=0x7ffb580fa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f9530 0x7ffb580fa5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ffb500048f0 tx=0x7ffb50009cb0).stop 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 shutdown_connections 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ffb440328f0 0x7ffb44034db0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb580f9530 0x7ffb580fa5d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb580f75b0 0x7ffb580f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 --2- 172.21.15.67:0/1272633384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb580f8c60 0x7ffb580fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 >> 172.21.15.67:0/1272633384 conn(0x7ffb5806dc60 msgr2=0x7ffb580fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 shutdown_connections 2024-02-10T08:16:32.712 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:32.709+0000 7ffb60260700 1 -- 172.21.15.67:0/1272633384 wait complete. 2024-02-10T08:16:33.018 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[24488]: audit 2024-02-10T08:16:32.709328+0000 mon.a (mon.0) 182 : audit [DBG] from='client.? 172.21.15.67:0/1272633384' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:33.019 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[24488]: audit 2024-02-10T08:16:32.796712+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:33.019 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[24488]: audit 2024-02-10T08:16:32.798046+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:33.019 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[24488]: audit 2024-02-10T08:16:32.799029+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:33.019 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[24488]: audit 2024-02-10T08:16:32.799992+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:33.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[17385]: audit 2024-02-10T08:16:32.709328+0000 mon.a (mon.0) 182 : audit [DBG] from='client.? 172.21.15.67:0/1272633384' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:33.019 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[17385]: audit 2024-02-10T08:16:32.796712+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:33.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[17385]: audit 2024-02-10T08:16:32.798046+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:33.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[17385]: audit 2024-02-10T08:16:32.799029+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:33.020 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:32 smithi067 bash[17385]: audit 2024-02-10T08:16:32.799992+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:33.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:32 smithi134 bash[18722]: audit 2024-02-10T08:16:32.709328+0000 mon.a (mon.0) 182 : audit [DBG] from='client.? 172.21.15.67:0/1272633384' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:33.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:32 smithi134 bash[18722]: audit 2024-02-10T08:16:32.796712+0000 mon.a (mon.0) 183 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:33.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:32 smithi134 bash[18722]: audit 2024-02-10T08:16:32.798046+0000 mon.a (mon.0) 184 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T08:16:33.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:32 smithi134 bash[18722]: audit 2024-02-10T08:16:32.799029+0000 mon.a (mon.0) 185 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T08:16:33.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:32 smithi134 bash[18722]: audit 2024-02-10T08:16:32.799992+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:33.518 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-10T08:16:33.518 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:16:33.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T08:16:33.531 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:16:33.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:16:33.588 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:16:33.588 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T08:16:33.600 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:16:33.600 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:16:33.656 INFO:tasks.cephadm:Adding mgr.y on smithi067 2024-02-10T08:16:33.657 INFO:tasks.cephadm:Adding mgr.x on smithi134 2024-02-10T08:16:33.657 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply mgr '2;smithi067=y;smithi134=x' 2024-02-10T08:16:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:33 smithi134 bash[18722]: cephadm 2024-02-10T08:16:32.797695+0000 mgr.y (mgr.14146) 67 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-10T08:16:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:33 smithi134 bash[18722]: cephadm 2024-02-10T08:16:32.800823+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Reconfiguring daemon mon.b on smithi134 2024-02-10T08:16:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:33 smithi134 bash[18722]: cluster 2024-02-10T08:16:32.824121+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:33 smithi067 bash[17385]: cephadm 2024-02-10T08:16:32.797695+0000 mgr.y (mgr.14146) 67 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-10T08:16:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:33 smithi067 bash[17385]: cephadm 2024-02-10T08:16:32.800823+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Reconfiguring daemon mon.b on smithi134 2024-02-10T08:16:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:33 smithi067 bash[17385]: cluster 2024-02-10T08:16:32.824121+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:33 smithi067 bash[24488]: cephadm 2024-02-10T08:16:32.797695+0000 mgr.y (mgr.14146) 67 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-10T08:16:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:33 smithi067 bash[24488]: cephadm 2024-02-10T08:16:32.800823+0000 mgr.y (mgr.14146) 68 : cephadm [INF] Reconfiguring daemon mon.b on smithi134 2024-02-10T08:16:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:33 smithi067 bash[24488]: cluster 2024-02-10T08:16:32.824121+0000 mgr.y (mgr.14146) 69 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:35 smithi134 bash[18722]: cluster 2024-02-10T08:16:34.824538+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:35 smithi134 bash[18722]: audit 2024-02-10T08:16:35.311705+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:35 smithi134 bash[18722]: audit 2024-02-10T08:16:35.313270+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:35 smithi134 bash[18722]: audit 2024-02-10T08:16:35.315908+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:35 smithi134 bash[18722]: audit 2024-02-10T08:16:35.317331+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:35 smithi134 bash[18722]: audit 2024-02-10T08:16:35.326743+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[17385]: cluster 2024-02-10T08:16:34.824538+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[17385]: audit 2024-02-10T08:16:35.311705+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[17385]: audit 2024-02-10T08:16:35.313270+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[17385]: audit 2024-02-10T08:16:35.315908+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[17385]: audit 2024-02-10T08:16:35.317331+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[17385]: audit 2024-02-10T08:16:35.326743+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[24488]: cluster 2024-02-10T08:16:34.824538+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[24488]: audit 2024-02-10T08:16:35.311705+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[24488]: audit 2024-02-10T08:16:35.313270+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[24488]: audit 2024-02-10T08:16:35.315908+0000 mon.a (mon.0) 189 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[24488]: audit 2024-02-10T08:16:35.317331+0000 mon.a (mon.0) 190 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:35 smithi067 bash[24488]: audit 2024-02-10T08:16:35.326743+0000 mon.a (mon.0) 191 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:37.098 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 -- 172.21.15.134:0/2296742295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f8de0 msgr2=0x7fdd800f9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 --2- 172.21.15.134:0/2296742295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f8de0 0x7fdd800f9240 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fdd78009c10 tx=0x7fdd78004d10).stop 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 -- 172.21.15.134:0/2296742295 shutdown_connections 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 --2- 172.21.15.134:0/2296742295 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f7b40 0x7fdd800f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 --2- 172.21.15.134:0/2296742295 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdd800f9730 0x7fdd80100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 --2- 172.21.15.134:0/2296742295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f8de0 0x7fdd800f9240 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 -- 172.21.15.134:0/2296742295 >> 172.21.15.134:0/2296742295 conn(0x7fdd800f3340 msgr2=0x7fdd800f5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 -- 172.21.15.134:0/2296742295 shutdown_connections 2024-02-10T08:16:37.099 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.091+0000 7fdd85455700 1 -- 172.21.15.134:0/2296742295 wait complete. 2024-02-10T08:16:37.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 Processor -- start 2024-02-10T08:16:37.100 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 -- start start 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdd800f7b40 0x7fdd8006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f8de0 0x7fdd8006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f9730 0x7fdd8006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8006bbe0 con 0x7fdd800f8de0 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8006bd50 con 0x7fdd800f9730 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8006bf10 con 0x7fdd800f7b40 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f9730 0x7fdd8006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:37.101 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f8de0 0x7fdd8006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:37.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f8de0 0x7fdd8006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.134:59138/0 (socket says 172.21.15.134:59138) 2024-02-10T08:16:37.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7e7fc700 1 -- 172.21.15.134:0/3222893023 learned_addr learned my addr 172.21.15.134:0/3222893023 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:16:37.102 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7effd700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdd800f7b40 0x7fdd8006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 -- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdd800f7b40 msgr2=0x7fdd8006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdd800f7b40 0x7fdd8006ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 -- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f8de0 msgr2=0x7fdd8006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f8de0 0x7fdd8006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 -- 172.21.15.134:0/3222893023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780097c0 con 0x7fdd800f9730 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd7f7fe700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f9730 0x7fdd8006b5c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fdd70002780 tx=0x7fdd70009e90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70007b40 con 0x7fdd800f9730 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd80112070 con 0x7fdd800f9730 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801125e0 con 0x7fdd800f9730 2024-02-10T08:16:37.103 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fdd7000f040 con 0x7fdd800f9730 2024-02-10T08:16:37.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.095+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70014670 con 0x7fdd800f9730 2024-02-10T08:16:37.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.099+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80102690 con 0x7fdd800f9730 2024-02-10T08:16:37.104 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.099+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7fdd70007ca0 con 0x7fdd800f9730 2024-02-10T08:16:37.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.099+0000 7fdd67fff700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdd680328f0 0x7fdd68034db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:37.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.099+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7fdd70042c30 con 0x7fdd800f9730 2024-02-10T08:16:37.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.099+0000 7fdd7effd700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdd680328f0 0x7fdd68034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:37.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.099+0000 7fdd7effd700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdd680328f0 0x7fdd68034db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdd800f8c40 tx=0x7fdd7400b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:37.110 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.103+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdd7001bd40 con 0x7fdd800f9730 2024-02-10T08:16:37.305 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.299+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi067=y;smithi134=x", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd800fee90 con 0x7fdd680328f0 2024-02-10T08:16:37.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.315+0000 7fdd67fff700 1 -- 172.21.15.134:0/3222893023 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdd800fee90 con 0x7fdd680328f0 2024-02-10T08:16:37.319 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.315+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdd680328f0 msgr2=0x7fdd68034db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.315+0000 7fdd85455700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdd680328f0 0x7fdd68034db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fdd800f8c40 tx=0x7fdd7400b040).stop 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.315+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f9730 msgr2=0x7fdd8006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.315+0000 7fdd85455700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f9730 0x7fdd8006b5c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fdd70002780 tx=0x7fdd70009e90).stop 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 shutdown_connections 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdd680328f0 0x7fdd68034db0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.323 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd800f9730 0x7fdd8006b5c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdd800f7b40 0x7fdd8006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 --2- 172.21.15.134:0/3222893023 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdd800f8de0 0x7fdd8006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:37.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 >> 172.21.15.134:0/3222893023 conn(0x7fdd800f3340 msgr2=0x7fdd800f56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:37.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 shutdown_connections 2024-02-10T08:16:37.324 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:16:37.319+0000 7fdd85455700 1 -- 172.21.15.134:0/3222893023 wait complete. 2024-02-10T08:16:38.131 DEBUG:teuthology.orchestra.run.smithi134:mgr.x> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.x.service 2024-02-10T08:16:38.134 INFO:tasks.cephadm:Deploying OSDs... 2024-02-10T08:16:38.134 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:16:38.134 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T08:16:38.142 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T08:16:38.142 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-02-10T08:16:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T08:16:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:38.194 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 495 Links: 1 2024-02-10T08:16:38.194 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:38.194 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-10 08:09:23.418908912 +0000 2024-02-10T08:16:38.194 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-10 08:09:23.166914724 +0000 2024-02-10T08:16:38.194 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-10 08:09:23.166914724 +0000 2024-02-10T08:16:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2024-02-10T08:16:38.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T08:16:38.249 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-10T08:16:38.249 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-10T08:16:38.249 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000418545 s, 1.2 MB/s 2024-02-10T08:16:38.250 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: cluster 2024-02-10T08:16:36.824949+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.318688+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.348573+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.351935+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.353504+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.363318+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.365303+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T08:16:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.372714+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T08:16:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.374039+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T08:16:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:38 smithi134 bash[18722]: audit 2024-02-10T08:16:37.375703+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:38.303 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-02-10T08:16:38.353 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 499 Links: 1 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-10 08:09:23.930897104 +0000 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-10 08:09:23.642903747 +0000 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-10 08:09:23.642903747 +0000 2024-02-10T08:16:38.354 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2024-02-10T08:16:38.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T08:16:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: cluster 2024-02-10T08:16:36.824949+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.318688+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.348573+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.351935+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.353504+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.363318+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.365303+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.372714+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.374039+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[17385]: audit 2024-02-10T08:16:37.375703+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: cluster 2024-02-10T08:16:36.824949+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.318688+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.348573+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.351935+0000 mon.a (mon.0) 194 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.353504+0000 mon.a (mon.0) 195 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:38.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.363318+0000 mon.a (mon.0) 196 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:38.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.365303+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T08:16:38.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.372714+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T08:16:38.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.374039+0000 mon.a (mon.0) 199 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T08:16:38.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:38 smithi067 bash[24488]: audit 2024-02-10T08:16:37.375703+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:38.385 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-10T08:16:38.385 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-10T08:16:38.386 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000432564 s, 1.2 MB/s 2024-02-10T08:16:38.387 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T08:16:38.443 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-02-10T08:16:38.493 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T08:16:38.493 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:38.493 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 505 Links: 1 2024-02-10T08:16:38.493 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:38.493 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-10 08:09:24.366887051 +0000 2024-02-10T08:16:38.494 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-10 08:09:24.126892585 +0000 2024-02-10T08:16:38.494 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-10 08:09:24.126892585 +0000 2024-02-10T08:16:38.494 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2024-02-10T08:16:38.494 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T08:16:38.549 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-10T08:16:38.549 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-10T08:16:38.549 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000413711 s, 1.2 MB/s 2024-02-10T08:16:38.550 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T08:16:38.573 INFO:journalctl@ceph.mgr.x.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:16:38.603 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-02-10T08:16:38.657 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T08:16:38.657 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:38.657 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 511 Links: 1 2024-02-10T08:16:38.658 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:38.658 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-10 08:09:24.858875705 +0000 2024-02-10T08:16:38.658 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-10 08:09:24.578882162 +0000 2024-02-10T08:16:38.658 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-10 08:09:24.578882162 +0000 2024-02-10T08:16:38.658 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: - 2024-02-10T08:16:38.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T08:16:38.717 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-10T08:16:38.717 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-10T08:16:38.717 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000375715 s, 1.4 MB/s 2024-02-10T08:16:38.718 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T08:16:38.771 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:16:38.771 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T08:16:38.779 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T08:16:38.779 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-02-10T08:16:38.830 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T08:16:38.830 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:38.830 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 496 Links: 1 2024-02-10T08:16:38.830 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:38.831 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-02-10 08:09:22.806297575 +0000 2024-02-10T08:16:38.831 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-02-10 08:09:22.534303847 +0000 2024-02-10T08:16:38.831 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-02-10 08:09:22.534303847 +0000 2024-02-10T08:16:38.831 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-02-10T08:16:38.831 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T08:16:38.888 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-02-10T08:16:38.888 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-02-10T08:16:38.888 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000298313 s, 1.7 MB/s 2024-02-10T08:16:38.889 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T08:16:38.939 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 502 Links: 1 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-02-10 08:09:23.314285860 +0000 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-02-10 08:09:23.034292317 +0000 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-02-10 08:09:23.034292317 +0000 2024-02-10T08:16:38.990 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-02-10T08:16:38.991 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T08:16:39.044 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-02-10T08:16:39.044 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-02-10T08:16:39.045 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000280178 s, 1.8 MB/s 2024-02-10T08:16:39.045 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T08:16:39.094 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-02-10T08:16:39.145 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T08:16:39.145 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:39.145 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 505 Links: 1 2024-02-10T08:16:39.146 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:39.146 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-02-10 08:09:23.518281156 +0000 2024-02-10T08:16:39.146 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-02-10 08:09:23.514281248 +0000 2024-02-10T08:16:39.146 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-02-10 08:09:23.514281248 +0000 2024-02-10T08:16:39.146 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-02-10T08:16:39.146 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T08:16:39.199 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:39 smithi134 bash[18722]: audit 2024-02-10T08:16:37.307532+0000 mgr.y (mgr.14146) 72 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi067=y;smithi134=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:39.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:39 smithi134 bash[18722]: cephadm 2024-02-10T08:16:37.311153+0000 mgr.y (mgr.14146) 73 : cephadm [INF] Saving service mgr spec with placement smithi067=y;smithi134=x;count:2 2024-02-10T08:16:39.200 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:39 smithi134 bash[18722]: cephadm 2024-02-10T08:16:37.376912+0000 mgr.y (mgr.14146) 74 : cephadm [INF] Deploying daemon mgr.x on smithi134 2024-02-10T08:16:39.201 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-02-10T08:16:39.201 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-02-10T08:16:39.201 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000313379 s, 1.6 MB/s 2024-02-10T08:16:39.202 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T08:16:39.251 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 511 Links: 1 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-02-10 08:09:24.258264090 +0000 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-02-10 08:09:23.990270270 +0000 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-02-10 08:09:23.990270270 +0000 2024-02-10T08:16:39.302 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-02-10T08:16:39.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T08:16:39.355 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-02-10T08:16:39.356 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-02-10T08:16:39.356 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000373278 s, 1.4 MB/s 2024-02-10T08:16:39.357 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T08:16:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:39 smithi067 bash[17385]: audit 2024-02-10T08:16:37.307532+0000 mgr.y (mgr.14146) 72 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi067=y;smithi134=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:39 smithi067 bash[17385]: cephadm 2024-02-10T08:16:37.311153+0000 mgr.y (mgr.14146) 73 : cephadm [INF] Saving service mgr spec with placement smithi067=y;smithi134=x;count:2 2024-02-10T08:16:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:39 smithi067 bash[17385]: cephadm 2024-02-10T08:16:37.376912+0000 mgr.y (mgr.14146) 74 : cephadm [INF] Deploying daemon mgr.x on smithi134 2024-02-10T08:16:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:39 smithi067 bash[24488]: audit 2024-02-10T08:16:37.307532+0000 mgr.y (mgr.14146) 72 : audit [DBG] from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi067=y;smithi134=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:39 smithi067 bash[24488]: cephadm 2024-02-10T08:16:37.311153+0000 mgr.y (mgr.14146) 73 : cephadm [INF] Saving service mgr spec with placement smithi067=y;smithi134=x;count:2 2024-02-10T08:16:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:39 smithi067 bash[24488]: cephadm 2024-02-10T08:16:37.376912+0000 mgr.y (mgr.14146) 74 : cephadm [INF] Deploying daemon mgr.x on smithi134 2024-02-10T08:16:39.408 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/vg_nvme/lv_4... 2024-02-10T08:16:39.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-10T08:16:39.874 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:39 smithi134 systemd[1]: Started Ceph mgr.x for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:16:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:40 smithi134 bash[18722]: cluster 2024-02-10T08:16:38.825413+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:40 smithi134 bash[18722]: audit 2024-02-10T08:16:39.851528+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:40 smithi134 bash[18722]: audit 2024-02-10T08:16:39.855302+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:40 smithi067 bash[17385]: cluster 2024-02-10T08:16:38.825413+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:40 smithi067 bash[17385]: audit 2024-02-10T08:16:39.851528+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:40 smithi067 bash[17385]: audit 2024-02-10T08:16:39.855302+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:40 smithi067 bash[24488]: cluster 2024-02-10T08:16:38.825413+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:40 smithi067 bash[24488]: audit 2024-02-10T08:16:39.851528+0000 mon.a (mon.0) 201 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:40 smithi067 bash[24488]: audit 2024-02-10T08:16:39.855302+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:40.989 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:40 smithi134 bash[19742]: debug 2024-02-10T08:16:40.819+0000 7f52f502e700 1 -- 172.21.15.134:0/83159782 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562546c44e0 con 0x5562547b8800 2024-02-10T08:16:41.280 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:40 smithi134 bash[19742]: debug 2024-02-10T08:16:40.983+0000 7f530562a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T08:16:41.280 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:41 smithi134 bash[19742]: debug 2024-02-10T08:16:41.087+0000 7f530562a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T08:16:42.022 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:41 smithi134 bash[19742]: debug 2024-02-10T08:16:41.551+0000 7f530562a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T08:16:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:42 smithi134 bash[18722]: cluster 2024-02-10T08:16:40.825722+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:42 smithi134 bash[18722]: audit 2024-02-10T08:16:41.812494+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:42 smithi067 bash[17385]: cluster 2024-02-10T08:16:40.825722+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:42 smithi067 bash[17385]: audit 2024-02-10T08:16:41.812494+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:42 smithi067 bash[24488]: cluster 2024-02-10T08:16:40.825722+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:42 smithi067 bash[24488]: audit 2024-02-10T08:16:41.812494+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:42.777 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:42 smithi134 bash[19742]: debug 2024-02-10T08:16:42.467+0000 7f530562a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T08:16:42.777 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:42 smithi134 bash[19742]: debug 2024-02-10T08:16:42.563+0000 7f530562a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134725 s, 778 MB/s 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 2024-02-10T08:16:42.780 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-02-10T08:16:43.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:42 smithi134 bash[19742]: debug 2024-02-10T08:16:42.811+0000 7f530562a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T08:16:43.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-02-10T08:16:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:44 smithi067 bash[24488]: cluster 2024-02-10T08:16:42.826134+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:44 smithi067 bash[17385]: cluster 2024-02-10T08:16:42.826134+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:44.218 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:44 smithi134 bash[18722]: cluster 2024-02-10T08:16:42.826134+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:44.218 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:43 smithi134 bash[19742]: debug 2024-02-10T08:16:43.947+0000 7f530562a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T08:16:44.218 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:44 smithi134 bash[19742]: debug 2024-02-10T08:16:44.111+0000 7f530562a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:16:44.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:44 smithi134 bash[19742]: debug 2024-02-10T08:16:44.211+0000 7f530562a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T08:16:44.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:44 smithi134 bash[19742]: debug 2024-02-10T08:16:44.323+0000 7f530562a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T08:16:44.957 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:44 smithi134 bash[19742]: debug 2024-02-10T08:16:44.551+0000 7f530562a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T08:16:45.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:44 smithi134 bash[19742]: debug 2024-02-10T08:16:44.951+0000 7f530562a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T08:16:45.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[19742]: debug 2024-02-10T08:16:45.083+0000 7f530562a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: cluster 2024-02-10T08:16:44.826539+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.856201+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.857613+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.858811+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.865061+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: cephadm 2024-02-10T08:16:44.867662+0000 mgr.y (mgr.14146) 79 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-10T08:16:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.868162+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.869606+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: audit 2024-02-10T08:16:44.870803+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[24488]: cephadm 2024-02-10T08:16:44.871908+0000 mgr.y (mgr.14146) 80 : cephadm [INF] Reconfiguring daemon mgr.y on smithi067 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: cluster 2024-02-10T08:16:44.826539+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.856201+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.857613+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.858811+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.865061+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: cephadm 2024-02-10T08:16:44.867662+0000 mgr.y (mgr.14146) 79 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.868162+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.869606+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: audit 2024-02-10T08:16:44.870803+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:45 smithi067 bash[17385]: cephadm 2024-02-10T08:16:44.871908+0000 mgr.y (mgr.14146) 80 : cephadm [INF] Reconfiguring daemon mgr.y on smithi067 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:46 smithi134 bash[19742]: debug 2024-02-10T08:16:46.015+0000 7f530562a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: cluster 2024-02-10T08:16:44.826539+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.856201+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.857613+0000 mon.a (mon.0) 205 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.858811+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.865061+0000 mon.a (mon.0) 207 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:46.128 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: cephadm 2024-02-10T08:16:44.867662+0000 mgr.y (mgr.14146) 79 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-10T08:16:46.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.868162+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T08:16:46.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.869606+0000 mon.a (mon.0) 209 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T08:16:46.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: audit 2024-02-10T08:16:44.870803+0000 mon.a (mon.0) 210 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:46.129 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:45 smithi134 bash[18722]: cephadm 2024-02-10T08:16:44.871908+0000 mgr.y (mgr.14146) 80 : cephadm [INF] Reconfiguring daemon mgr.y on smithi067 2024-02-10T08:16:46.487 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:46 smithi134 bash[19742]: debug 2024-02-10T08:16:46.123+0000 7f530562a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T08:16:46.487 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:46 smithi134 bash[19742]: debug 2024-02-10T08:16:46.243+0000 7f530562a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/2474217462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f7b00 msgr2=0x7f89240f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- 172.21.15.67:0/2474217462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f7b00 0x7f89240f7f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f8914009be0 tx=0x7f8914004d10).stop 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/2474217462 shutdown_connections 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- 172.21.15.67:0/2474217462 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f7b00 0x7f89240f7f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- 172.21.15.67:0/2474217462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89240f9520 0x7f8924100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- 172.21.15.67:0/2474217462 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f8410 0x7f89240f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/2474217462 >> 172.21.15.67:0/2474217462 conn(0x7f89240f3340 msgr2=0x7f89240f5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/2474217462 shutdown_connections 2024-02-10T08:16:47.060 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/2474217462 wait complete. 2024-02-10T08:16:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 Processor -- start 2024-02-10T08:16:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- start start 2024-02-10T08:16:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89240f7b00 0x7f8924109ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:47.062 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f8410 0x7f892410a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f9520 0x7f8924111490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924111980 con 0x7f89240f8410 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924111ac0 con 0x7f89240f7b00 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8924111c00 con 0x7f89240f9520 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8923fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f8410 0x7f892410a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f9520 0x7f8924111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8923fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f8410 0x7f892410a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50652/0 (socket says 172.21.15.67:50652) 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8923fff700 1 -- 172.21.15.67:0/46075236 learned_addr learned my addr 172.21.15.67:0/46075236 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8928cd3700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89240f7b00 0x7f8924109ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 -- 172.21.15.67:0/46075236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89240f7b00 msgr2=0x7f8924109ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89240f7b00 0x7f8924109ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 -- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f8410 msgr2=0x7f892410a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f8410 0x7f892410a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:16:47.063 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 -- 172.21.15.67:0/46075236 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89140097c0 con 0x7f89240f9520 2024-02-10T08:16:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f89294d4700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f9520 0x7f8924111490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f891c00b490 tx=0x7f891c006f30).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c012450 con 0x7f89240f9520 2024-02-10T08:16:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f891c012a40 con 0x7f89240f9520 2024-02-10T08:16:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8924111ee0 con 0x7f89240f9520 2024-02-10T08:16:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0119a0 con 0x7f89240f9520 2024-02-10T08:16:47.064 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.056+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8924112340 con 0x7f89240f9520 2024-02-10T08:16:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.060+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 13) v1 ==== 38673+0+0 (secure 0 0 0) 0x7f891c0125b0 con 0x7f89240f9520 2024-02-10T08:16:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.060+0000 7f8921ffb700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f89100328f0 0x7f8910034db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:16:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.060+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1098+0+0 (secure 0 0 0) 0x7f891c042ab0 con 0x7f89240f9520 2024-02-10T08:16:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.060+0000 7f8928cd3700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f89100328f0 0x7f8910034db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:16:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.060+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8924060070 con 0x7f89240f9520 2024-02-10T08:16:47.065 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.060+0000 7f8928cd3700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f89100328f0 0x7f8910034db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f89140048e0 tx=0x7f8914004d10).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:16:47.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.068+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f891c0192a0 con 0x7f89240f9520 2024-02-10T08:16:47.124 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:46 smithi134 bash[19742]: debug 2024-02-10T08:16:46.479+0000 7f530562a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T08:16:47.124 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:46 smithi134 bash[19742]: debug 2024-02-10T08:16:46.783+0000 7f530562a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T08:16:47.124 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:46 smithi134 bash[19742]: debug 2024-02-10T08:16:46.947+0000 7f530562a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T08:16:47.266 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:47.264+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f892406c6d0 con 0x7f89100328f0 2024-02-10T08:16:47.444 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[19742]: debug 2024-02-10T08:16:47.119+0000 7f530562a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:16:47.444 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[19742]: debug 2024-02-10T08:16:47.335+0000 7f530562a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T08:16:48.006 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:16:48.000+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f891c01cc90 con 0x7f89240f9520 2024-02-10T08:16:48.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[19742]: debug 2024-02-10T08:16:47.439+0000 7f530562a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T08:16:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: cluster 2024-02-10T08:16:46.826839+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.271024+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.276258+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.277214+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: cluster 2024-02-10T08:16:47.466137+0000 mon.a (mon.0) 214 : cluster [DBG] Standby manager daemon x started 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.469488+0000 mon.c (mon.2) 2 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.470231+0000 mon.c (mon.2) 3 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.472129+0000 mon.c (mon.2) 4 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.472893+0000 mon.c (mon.2) 5 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.697872+0000 mon.a (mon.0) 215 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.699822+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.702465+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.703723+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:48.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:47 smithi134 bash[18722]: audit 2024-02-10T08:16:47.710211+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: cluster 2024-02-10T08:16:46.826839+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.271024+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:16:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.276258+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.277214+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: cluster 2024-02-10T08:16:47.466137+0000 mon.a (mon.0) 214 : cluster [DBG] Standby manager daemon x started 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.469488+0000 mon.c (mon.2) 2 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.470231+0000 mon.c (mon.2) 3 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.472129+0000 mon.c (mon.2) 4 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.472893+0000 mon.c (mon.2) 5 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.697872+0000 mon.a (mon.0) 215 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.699822+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.702465+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.703723+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[17385]: audit 2024-02-10T08:16:47.710211+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: cluster 2024-02-10T08:16:46.826839+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.271024+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.276258+0000 mon.a (mon.0) 212 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.277214+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: cluster 2024-02-10T08:16:47.466137+0000 mon.a (mon.0) 214 : cluster [DBG] Standby manager daemon x started 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.469488+0000 mon.c (mon.2) 2 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.470231+0000 mon.c (mon.2) 3 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.472129+0000 mon.c (mon.2) 4 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.472893+0000 mon.c (mon.2) 5 : audit [DBG] from='mgr.? 172.21.15.134:0/2693811580' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.697872+0000 mon.a (mon.0) 215 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.699822+0000 mon.a (mon.0) 216 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.702465+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.703723+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:16:48.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:47 smithi067 bash[24488]: audit 2024-02-10T08:16:47.710211+0000 mon.a (mon.0) 219 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:16:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:49 smithi134 bash[18722]: audit 2024-02-10T08:16:47.267830+0000 mgr.y (mgr.14146) 82 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:49 smithi134 bash[18722]: cluster 2024-02-10T08:16:48.006777+0000 mon.a (mon.0) 220 : cluster [DBG] mgrmap e14: y(active, since 2m), standbys: x 2024-02-10T08:16:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:49 smithi134 bash[18722]: audit 2024-02-10T08:16:48.007185+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T08:16:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:49 smithi067 bash[17385]: audit 2024-02-10T08:16:47.267830+0000 mgr.y (mgr.14146) 82 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:49 smithi067 bash[17385]: cluster 2024-02-10T08:16:48.006777+0000 mon.a (mon.0) 220 : cluster [DBG] mgrmap e14: y(active, since 2m), standbys: x 2024-02-10T08:16:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:49 smithi067 bash[17385]: audit 2024-02-10T08:16:48.007185+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T08:16:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:49 smithi067 bash[24488]: audit 2024-02-10T08:16:47.267830+0000 mgr.y (mgr.14146) 82 : audit [DBG] from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:16:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:49 smithi067 bash[24488]: cluster 2024-02-10T08:16:48.006777+0000 mon.a (mon.0) 220 : cluster [DBG] mgrmap e14: y(active, since 2m), standbys: x 2024-02-10T08:16:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:49 smithi067 bash[24488]: audit 2024-02-10T08:16:48.007185+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T08:16:50.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:50 smithi134 bash[18722]: cluster 2024-02-10T08:16:48.827211+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:50 smithi067 bash[17385]: cluster 2024-02-10T08:16:48.827211+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:50 smithi067 bash[24488]: cluster 2024-02-10T08:16:48.827211+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:52 smithi134 bash[18722]: cluster 2024-02-10T08:16:50.827683+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:52 smithi067 bash[17385]: cluster 2024-02-10T08:16:50.827683+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:52 smithi067 bash[24488]: cluster 2024-02-10T08:16:50.827683+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:54 smithi134 bash[18722]: cluster 2024-02-10T08:16:52.828136+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:54.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:54 smithi067 bash[24488]: cluster 2024-02-10T08:16:52.828136+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:54.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:54 smithi067 bash[17385]: cluster 2024-02-10T08:16:52.828136+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[17385]: cluster 2024-02-10T08:16:54.828557+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[17385]: audit 2024-02-10T08:16:55.453074+0000 mon.a (mon.0) 222 : audit [INF] from='client.? 172.21.15.67:0/2126442015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437dcceb-1970-491f-b22e-be8d9b063290"}]: dispatch 2024-02-10T08:16:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[17385]: audit 2024-02-10T08:16:55.460793+0000 mon.a (mon.0) 223 : audit [INF] from='client.? 172.21.15.67:0/2126442015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437dcceb-1970-491f-b22e-be8d9b063290"}]': finished 2024-02-10T08:16:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[17385]: cluster 2024-02-10T08:16:55.460851+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-10T08:16:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[17385]: audit 2024-02-10T08:16:55.461013+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:16:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[24488]: cluster 2024-02-10T08:16:54.828557+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[24488]: audit 2024-02-10T08:16:55.453074+0000 mon.a (mon.0) 222 : audit [INF] from='client.? 172.21.15.67:0/2126442015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437dcceb-1970-491f-b22e-be8d9b063290"}]: dispatch 2024-02-10T08:16:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[24488]: audit 2024-02-10T08:16:55.460793+0000 mon.a (mon.0) 223 : audit [INF] from='client.? 172.21.15.67:0/2126442015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437dcceb-1970-491f-b22e-be8d9b063290"}]': finished 2024-02-10T08:16:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[24488]: cluster 2024-02-10T08:16:55.460851+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-10T08:16:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:56 smithi067 bash[24488]: audit 2024-02-10T08:16:55.461013+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:16:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:56 smithi134 bash[18722]: cluster 2024-02-10T08:16:54.828557+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:56 smithi134 bash[18722]: audit 2024-02-10T08:16:55.453074+0000 mon.a (mon.0) 222 : audit [INF] from='client.? 172.21.15.67:0/2126442015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "437dcceb-1970-491f-b22e-be8d9b063290"}]: dispatch 2024-02-10T08:16:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:56 smithi134 bash[18722]: audit 2024-02-10T08:16:55.460793+0000 mon.a (mon.0) 223 : audit [INF] from='client.? 172.21.15.67:0/2126442015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "437dcceb-1970-491f-b22e-be8d9b063290"}]': finished 2024-02-10T08:16:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:56 smithi134 bash[18722]: cluster 2024-02-10T08:16:55.460851+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-10T08:16:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:56 smithi134 bash[18722]: audit 2024-02-10T08:16:55.461013+0000 mon.a (mon.0) 225 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:16:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:57 smithi067 bash[17385]: audit 2024-02-10T08:16:56.165267+0000 mon.a (mon.0) 226 : audit [DBG] from='client.? 172.21.15.67:0/231107997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:16:57.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:57 smithi067 bash[24488]: audit 2024-02-10T08:16:56.165267+0000 mon.a (mon.0) 226 : audit [DBG] from='client.? 172.21.15.67:0/231107997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:16:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:57 smithi134 bash[18722]: audit 2024-02-10T08:16:56.165267+0000 mon.a (mon.0) 226 : audit [DBG] from='client.? 172.21.15.67:0/231107997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:16:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:16:58 smithi067 bash[17385]: cluster 2024-02-10T08:16:56.828962+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:58.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:16:58 smithi067 bash[24488]: cluster 2024-02-10T08:16:56.828962+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:16:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:16:58 smithi134 bash[18722]: cluster 2024-02-10T08:16:56.828962+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:00 smithi067 bash[17385]: cluster 2024-02-10T08:16:58.829399+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:00 smithi067 bash[24488]: cluster 2024-02-10T08:16:58.829399+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:00 smithi134 bash[18722]: cluster 2024-02-10T08:16:58.829399+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:01 smithi067 bash[17385]: cluster 2024-02-10T08:17:00.829836+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:01.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:01 smithi067 bash[24488]: cluster 2024-02-10T08:17:00.829836+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:01 smithi134 bash[18722]: cluster 2024-02-10T08:17:00.829836+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:04.251 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:03 smithi067 bash[17385]: cluster 2024-02-10T08:17:02.830277+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:04.251 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:04 smithi067 bash[24488]: cluster 2024-02-10T08:17:02.830277+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:04 smithi134 bash[18722]: cluster 2024-02-10T08:17:02.830277+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:06 smithi134 bash[18722]: cluster 2024-02-10T08:17:04.830737+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:06 smithi067 bash[17385]: cluster 2024-02-10T08:17:04.830737+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:06 smithi067 bash[24488]: cluster 2024-02-10T08:17:04.830737+0000 mgr.y (mgr.14146) 91 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:08.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:08 smithi134 bash[18722]: cluster 2024-02-10T08:17:06.831124+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:08.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:08 smithi134 bash[18722]: audit 2024-02-10T08:17:07.996720+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T08:17:08.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:08 smithi134 bash[18722]: audit 2024-02-10T08:17:07.998432+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:08 smithi067 bash[17385]: cluster 2024-02-10T08:17:06.831124+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:08 smithi067 bash[17385]: audit 2024-02-10T08:17:07.996720+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T08:17:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:08 smithi067 bash[17385]: audit 2024-02-10T08:17:07.998432+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:08.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:08 smithi067 bash[24488]: cluster 2024-02-10T08:17:06.831124+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:08.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:08 smithi067 bash[24488]: audit 2024-02-10T08:17:07.996720+0000 mon.a (mon.0) 227 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T08:17:08.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:08 smithi067 bash[24488]: audit 2024-02-10T08:17:07.998432+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:09 smithi134 bash[18722]: cephadm 2024-02-10T08:17:07.999800+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Deploying daemon osd.0 on smithi067 2024-02-10T08:17:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:09 smithi067 bash[17385]: cephadm 2024-02-10T08:17:07.999800+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Deploying daemon osd.0 on smithi067 2024-02-10T08:17:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:09 smithi067 bash[24488]: cephadm 2024-02-10T08:17:07.999800+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Deploying daemon osd.0 on smithi067 2024-02-10T08:17:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:10 smithi134 bash[18722]: cluster 2024-02-10T08:17:08.831671+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:10 smithi067 bash[17385]: cluster 2024-02-10T08:17:08.831671+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:10 smithi067 bash[24488]: cluster 2024-02-10T08:17:08.831671+0000 mgr.y (mgr.14146) 94 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:12 smithi134 bash[18722]: cluster 2024-02-10T08:17:10.832155+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:12 smithi067 bash[24488]: cluster 2024-02-10T08:17:10.832155+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:12 smithi067 bash[17385]: cluster 2024-02-10T08:17:10.832155+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:13 smithi067 bash[17385]: audit 2024-02-10T08:17:12.431381+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:13 smithi067 bash[17385]: audit 2024-02-10T08:17:12.442519+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:13 smithi067 bash[17385]: cluster 2024-02-10T08:17:12.832625+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:13 smithi067 bash[24488]: audit 2024-02-10T08:17:12.431381+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:13 smithi067 bash[24488]: audit 2024-02-10T08:17:12.442519+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:13 smithi067 bash[24488]: cluster 2024-02-10T08:17:12.832625+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:13 smithi134 bash[18722]: audit 2024-02-10T08:17:12.431381+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:13 smithi134 bash[18722]: audit 2024-02-10T08:17:12.442519+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:13 smithi134 bash[18722]: cluster 2024-02-10T08:17:12.832625+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:15 smithi134 bash[18722]: cluster 2024-02-10T08:17:14.833098+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:15 smithi067 bash[17385]: cluster 2024-02-10T08:17:14.833098+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:16.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:15 smithi067 bash[24488]: cluster 2024-02-10T08:17:14.833098+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:18 smithi134 bash[18722]: cluster 2024-02-10T08:17:16.833528+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:18 smithi134 bash[18722]: audit 2024-02-10T08:17:17.540889+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:18 smithi134 bash[18722]: audit 2024-02-10T08:17:17.809365+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:18 smithi134 bash[18722]: audit 2024-02-10T08:17:17.810375+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:17:18.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:18 smithi134 bash[18722]: audit 2024-02-10T08:17:17.817148+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[17385]: cluster 2024-02-10T08:17:16.833528+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[17385]: audit 2024-02-10T08:17:17.540889+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[17385]: audit 2024-02-10T08:17:17.809365+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[17385]: audit 2024-02-10T08:17:17.810375+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:17:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[17385]: audit 2024-02-10T08:17:17.817148+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:18.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[24488]: cluster 2024-02-10T08:17:16.833528+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:18.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[24488]: audit 2024-02-10T08:17:17.540889+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:18.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[24488]: audit 2024-02-10T08:17:17.809365+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:18.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[24488]: audit 2024-02-10T08:17:17.810375+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:17:18.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:18 smithi067 bash[24488]: audit 2024-02-10T08:17:17.817148+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:19.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:19 smithi134 bash[18722]: audit 2024-02-10T08:17:18.885965+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T08:17:19.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:19 smithi134 bash[18722]: audit 2024-02-10T08:17:18.886505+0000 mon.a (mon.0) 235 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T08:17:19.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:19 smithi067 bash[17385]: audit 2024-02-10T08:17:18.885965+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T08:17:19.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:19 smithi067 bash[17385]: audit 2024-02-10T08:17:18.886505+0000 mon.a (mon.0) 235 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T08:17:19.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:19 smithi067 bash[24488]: audit 2024-02-10T08:17:18.885965+0000 mon.c (mon.2) 6 : audit [INF] from='osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T08:17:19.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:19 smithi067 bash[24488]: audit 2024-02-10T08:17:18.886505+0000 mon.a (mon.0) 235 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T08:17:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[17385]: cluster 2024-02-10T08:17:18.833894+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[17385]: audit 2024-02-10T08:17:19.010805+0000 mon.a (mon.0) 236 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T08:17:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[17385]: cluster 2024-02-10T08:17:19.010917+0000 mon.a (mon.0) 237 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-10T08:17:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[17385]: audit 2024-02-10T08:17:19.011112+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[17385]: audit 2024-02-10T08:17:19.013460+0000 mon.c (mon.2) 7 : audit [INF] from='osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:17:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[17385]: audit 2024-02-10T08:17:19.013890+0000 mon.a (mon.0) 239 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:17:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[24488]: cluster 2024-02-10T08:17:18.833894+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[24488]: audit 2024-02-10T08:17:19.010805+0000 mon.a (mon.0) 236 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T08:17:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[24488]: cluster 2024-02-10T08:17:19.010917+0000 mon.a (mon.0) 237 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-10T08:17:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[24488]: audit 2024-02-10T08:17:19.011112+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[24488]: audit 2024-02-10T08:17:19.013460+0000 mon.c (mon.2) 7 : audit [INF] from='osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:17:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:20 smithi067 bash[24488]: audit 2024-02-10T08:17:19.013890+0000 mon.a (mon.0) 239 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:17:20.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:20 smithi134 bash[18722]: cluster 2024-02-10T08:17:18.833894+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:20.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:20 smithi134 bash[18722]: audit 2024-02-10T08:17:19.010805+0000 mon.a (mon.0) 236 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T08:17:20.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:20 smithi134 bash[18722]: cluster 2024-02-10T08:17:19.010917+0000 mon.a (mon.0) 237 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-10T08:17:20.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:20 smithi134 bash[18722]: audit 2024-02-10T08:17:19.011112+0000 mon.a (mon.0) 238 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:20.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:20 smithi134 bash[18722]: audit 2024-02-10T08:17:19.013460+0000 mon.c (mon.2) 7 : audit [INF] from='osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:17:20.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:20 smithi134 bash[18722]: audit 2024-02-10T08:17:19.013890+0000 mon.a (mon.0) 239 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:17:21.001 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.995+0000 7f8921ffb700 1 -- 172.21.15.67:0/46075236 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f892406c6d0 con 0x7f89100328f0 2024-02-10T08:17:21.001 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f89100328f0 msgr2=0x7f8910034db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f89100328f0 0x7f8910034db0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f89140048e0 tx=0x7f8914004d10).stop 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f9520 msgr2=0x7f8924111490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f9520 0x7f8924111490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f891c00b490 tx=0x7f891c006f30).stop 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 shutdown_connections 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f89100328f0 0x7f8910034db0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f89240f9520 0x7f8924111490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89240f7b00 0x7f8924109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 --2- 172.21.15.67:0/46075236 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f89240f8410 0x7f892410a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 >> 172.21.15.67:0/46075236 conn(0x7f89240f3340 msgr2=0x7f892406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:17:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 shutdown_connections 2024-02-10T08:17:21.005 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:20.999+0000 7f892af37700 1 -- 172.21.15.67:0/46075236 wait complete. 2024-02-10T08:17:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[17385]: audit 2024-02-10T08:17:20.014640+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:17:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[17385]: cluster 2024-02-10T08:17:20.014928+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-10T08:17:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[17385]: audit 2024-02-10T08:17:20.015629+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[17385]: audit 2024-02-10T08:17:20.022143+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[17385]: audit 2024-02-10T08:17:21.000047+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[24488]: audit 2024-02-10T08:17:20.014640+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:17:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[24488]: cluster 2024-02-10T08:17:20.014928+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-10T08:17:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[24488]: audit 2024-02-10T08:17:20.015629+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[24488]: audit 2024-02-10T08:17:20.022143+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:21 smithi067 bash[24488]: audit 2024-02-10T08:17:21.000047+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:21 smithi134 bash[18722]: audit 2024-02-10T08:17:20.014640+0000 mon.a (mon.0) 240 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:17:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:21 smithi134 bash[18722]: cluster 2024-02-10T08:17:20.014928+0000 mon.a (mon.0) 241 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-10T08:17:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:21 smithi134 bash[18722]: audit 2024-02-10T08:17:20.015629+0000 mon.a (mon.0) 242 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:21 smithi134 bash[18722]: audit 2024-02-10T08:17:20.022143+0000 mon.a (mon.0) 243 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:21 smithi134 bash[18722]: audit 2024-02-10T08:17:21.000047+0000 mon.a (mon.0) 244 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:21.713 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.0.service 2024-02-10T08:17:21.716 INFO:tasks.cephadm:Deploying osd.1 on smithi067 with /dev/vg_nvme/lv_3... 2024-02-10T08:17:21.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-10T08:17:21.943 INFO:journalctl@ceph.osd.0.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: cluster 2024-02-10T08:17:19.929812+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: cluster 2024-02-10T08:17:19.929874+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: cluster 2024-02-10T08:17:20.834328+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: cluster 2024-02-10T08:17:21.024540+0000 mon.a (mon.0) 245 : cluster [INF] osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] boot 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: cluster 2024-02-10T08:17:21.024653+0000 mon.a (mon.0) 246 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: audit 2024-02-10T08:17:21.025576+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[17385]: audit 2024-02-10T08:17:21.041876+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: cluster 2024-02-10T08:17:19.929812+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: cluster 2024-02-10T08:17:19.929874+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: cluster 2024-02-10T08:17:20.834328+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: cluster 2024-02-10T08:17:21.024540+0000 mon.a (mon.0) 245 : cluster [INF] osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] boot 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: cluster 2024-02-10T08:17:21.024653+0000 mon.a (mon.0) 246 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: audit 2024-02-10T08:17:21.025576+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:22 smithi067 bash[24488]: audit 2024-02-10T08:17:21.041876+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: cluster 2024-02-10T08:17:19.929812+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:17:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: cluster 2024-02-10T08:17:19.929874+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:17:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: cluster 2024-02-10T08:17:20.834328+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T08:17:22.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: cluster 2024-02-10T08:17:21.024540+0000 mon.a (mon.0) 245 : cluster [INF] osd.0 [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] boot 2024-02-10T08:17:22.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: cluster 2024-02-10T08:17:21.024653+0000 mon.a (mon.0) 246 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-10T08:17:22.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: audit 2024-02-10T08:17:21.025576+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:17:22.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:22 smithi134 bash[18722]: audit 2024-02-10T08:17:21.041876+0000 mon.a (mon.0) 248 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:23.304 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:23 smithi067 bash[17385]: cluster 2024-02-10T08:17:22.040343+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-10T08:17:23.304 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:23 smithi067 bash[24488]: cluster 2024-02-10T08:17:22.040343+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-10T08:17:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:23 smithi134 bash[18722]: cluster 2024-02-10T08:17:22.040343+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-10T08:17:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:24 smithi067 bash[17385]: cluster 2024-02-10T08:17:22.834861+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:24.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:24 smithi067 bash[24488]: cluster 2024-02-10T08:17:22.834861+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:24 smithi134 bash[18722]: cluster 2024-02-10T08:17:22.834861+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-10T08:17:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-10T08:17:25.344 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-02-10T08:17:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-02-10T08:17:25.345 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0160149 s, 655 MB/s 2024-02-10T08:17:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-02-10T08:17:26.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-02-10T08:17:26.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:26 smithi067 bash[24488]: cluster 2024-02-10T08:17:24.835311+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:26 smithi067 bash[17385]: cluster 2024-02-10T08:17:24.835311+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:26 smithi134 bash[18722]: cluster 2024-02-10T08:17:24.835311+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:28.276 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:28 smithi067 bash[24488]: cluster 2024-02-10T08:17:26.835768+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:28.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:28 smithi067 bash[17385]: cluster 2024-02-10T08:17:26.835768+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:28.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:28 smithi134 bash[18722]: cluster 2024-02-10T08:17:26.835768+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- 172.21.15.67:0/4166989398 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400fb530 msgr2=0x7f6d400fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:17:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- 172.21.15.67:0/4166989398 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400fb530 0x7f6d400fb9b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f6d38004870 tx=0x7f6d38009cb0).stop 2024-02-10T08:17:29.661 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- 172.21.15.67:0/4166989398 shutdown_connections 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- 172.21.15.67:0/4166989398 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d400f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- 172.21.15.67:0/4166989398 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400fb530 0x7f6d400fb9b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- 172.21.15.67:0/4166989398 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400f8a80 0x7f6d400faf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- 172.21.15.67:0/4166989398 >> 172.21.15.67:0/4166989398 conn(0x7f6d400f2230 msgr2=0x7f6d400f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- 172.21.15.67:0/4166989398 shutdown_connections 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- 172.21.15.67:0/4166989398 wait complete. 2024-02-10T08:17:29.662 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 Processor -- start 2024-02-10T08:17:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- start start 2024-02-10T08:17:29.663 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d4010e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400f8a80 0x7f6d4010e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400fb530 0x7f6d40115900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40115df0 con 0x7f6d40061770 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40115f30 con 0x7f6d400fb530 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d4836b700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40116070 con 0x7f6d400f8a80 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d46107700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d4010e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d46107700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d4010e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54276/0 (socket says 172.21.15.67:54276) 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d46107700 1 -- 172.21.15.67:0/697927284 learned_addr learned my addr 172.21.15.67:0/697927284 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.655+0000 7f6d45906700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400f8a80 0x7f6d4010e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46908700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400fb530 0x7f6d40115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46107700 1 -- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400f8a80 msgr2=0x7f6d4010e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:17:29.664 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46107700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400f8a80 0x7f6d4010e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46107700 1 -- 172.21.15.67:0/697927284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400fb530 msgr2=0x7f6d40115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46107700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400fb530 0x7f6d40115900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46107700 1 -- 172.21.15.67:0/697927284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d38005040 con 0x7f6d40061770 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d46107700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d4010e330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6d3c00eca0 tx=0x7f6d3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c009930 con 0x7f6d40061770 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f6d3c00f040 con 0x7f6d40061770 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00c9c0 con 0x7f6d40061770 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d40116350 con 0x7f6d40061770 2024-02-10T08:17:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d40116840 con 0x7f6d40061770 2024-02-10T08:17:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4005f400 con 0x7f6d40061770 2024-02-10T08:17:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f6d3c0092e0 con 0x7f6d40061770 2024-02-10T08:17:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d377fe700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f6d2c061250 0x7f6d2c063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:17:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1710+0+0 (secure 0 0 0) 0x7f6d3c07b5a0 con 0x7f6d40061770 2024-02-10T08:17:29.666 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.659+0000 7f6d45906700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f6d2c061250 0x7f6d2c063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:17:29.667 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.663+0000 7f6d45906700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f6d2c061250 0x7f6d2c063710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6d30006fd0 tx=0x7f6d30009060).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:17:29.676 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.671+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6d3c04af60 con 0x7f6d40061770 2024-02-10T08:17:29.866 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:17:29.859+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d400591c0 con 0x7f6d2c061250 2024-02-10T08:17:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:30 smithi134 bash[18722]: cluster 2024-02-10T08:17:28.836287+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:30 smithi134 bash[18722]: audit 2024-02-10T08:17:29.869824+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:17:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:30 smithi134 bash[18722]: audit 2024-02-10T08:17:29.872824+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:17:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:30 smithi134 bash[18722]: audit 2024-02-10T08:17:29.873611+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[24488]: cluster 2024-02-10T08:17:28.836287+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[24488]: audit 2024-02-10T08:17:29.869824+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[24488]: audit 2024-02-10T08:17:29.872824+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[24488]: audit 2024-02-10T08:17:29.873611+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[17385]: cluster 2024-02-10T08:17:28.836287+0000 mgr.y (mgr.14146) 104 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[17385]: audit 2024-02-10T08:17:29.869824+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[17385]: audit 2024-02-10T08:17:29.872824+0000 mon.a (mon.0) 251 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:17:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:30 smithi067 bash[17385]: audit 2024-02-10T08:17:29.873611+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:31 smithi134 bash[18722]: audit 2024-02-10T08:17:29.867093+0000 mgr.y (mgr.14146) 105 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:17:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:31 smithi067 bash[17385]: audit 2024-02-10T08:17:29.867093+0000 mgr.y (mgr.14146) 105 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:17:31.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:31 smithi067 bash[24488]: audit 2024-02-10T08:17:29.867093+0000 mgr.y (mgr.14146) 105 : audit [DBG] from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:17:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:32 smithi134 bash[18722]: cluster 2024-02-10T08:17:30.836749+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:32 smithi067 bash[17385]: cluster 2024-02-10T08:17:30.836749+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:32 smithi067 bash[24488]: cluster 2024-02-10T08:17:30.836749+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: cluster 2024-02-10T08:17:32.837134+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: cephadm 2024-02-10T08:17:32.946376+0000 mgr.y (mgr.14146) 108 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: audit 2024-02-10T08:17:32.953697+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: audit 2024-02-10T08:17:32.955224+0000 mon.a (mon.0) 254 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: cephadm 2024-02-10T08:17:32.956183+0000 mgr.y (mgr.14146) 109 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 16241M 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: audit 2024-02-10T08:17:32.963612+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: audit 2024-02-10T08:17:32.965224+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: audit 2024-02-10T08:17:32.966343+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:17:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:33 smithi134 bash[18722]: audit 2024-02-10T08:17:32.975296+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: cluster 2024-02-10T08:17:32.837134+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: cephadm 2024-02-10T08:17:32.946376+0000 mgr.y (mgr.14146) 108 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:17:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: audit 2024-02-10T08:17:32.953697+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: audit 2024-02-10T08:17:32.955224+0000 mon.a (mon.0) 254 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:17:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: cephadm 2024-02-10T08:17:32.956183+0000 mgr.y (mgr.14146) 109 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 16241M 2024-02-10T08:17:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: audit 2024-02-10T08:17:32.963612+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: audit 2024-02-10T08:17:32.965224+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: audit 2024-02-10T08:17:32.966343+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[17385]: audit 2024-02-10T08:17:32.975296+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: cluster 2024-02-10T08:17:32.837134+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: cephadm 2024-02-10T08:17:32.946376+0000 mgr.y (mgr.14146) 108 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: audit 2024-02-10T08:17:32.953697+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: audit 2024-02-10T08:17:32.955224+0000 mon.a (mon.0) 254 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: cephadm 2024-02-10T08:17:32.956183+0000 mgr.y (mgr.14146) 109 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 16241M 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: audit 2024-02-10T08:17:32.963612+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: audit 2024-02-10T08:17:32.965224+0000 mon.a (mon.0) 256 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: audit 2024-02-10T08:17:32.966343+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:17:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:33 smithi067 bash[24488]: audit 2024-02-10T08:17:32.975296+0000 mon.a (mon.0) 258 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:36.256 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:35 smithi067 bash[17385]: cluster 2024-02-10T08:17:34.837598+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:36.256 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:35 smithi067 bash[24488]: cluster 2024-02-10T08:17:34.837598+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:35 smithi134 bash[18722]: cluster 2024-02-10T08:17:34.837598+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:38 smithi067 bash[17385]: cluster 2024-02-10T08:17:36.838063+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:38 smithi067 bash[24488]: cluster 2024-02-10T08:17:36.838063+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:38 smithi134 bash[18722]: cluster 2024-02-10T08:17:36.838063+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:39 smithi134 bash[18722]: audit 2024-02-10T08:17:38.763135+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.67:0/3958496578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]: dispatch 2024-02-10T08:17:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:39 smithi134 bash[18722]: audit 2024-02-10T08:17:38.763769+0000 mon.a (mon.0) 259 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]: dispatch 2024-02-10T08:17:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:39 smithi134 bash[18722]: audit 2024-02-10T08:17:38.771156+0000 mon.a (mon.0) 260 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]': finished 2024-02-10T08:17:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:39 smithi134 bash[18722]: cluster 2024-02-10T08:17:38.771222+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-10T08:17:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:39 smithi134 bash[18722]: audit 2024-02-10T08:17:38.771366+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[24488]: audit 2024-02-10T08:17:38.763135+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.67:0/3958496578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]: dispatch 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[24488]: audit 2024-02-10T08:17:38.763769+0000 mon.a (mon.0) 259 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]: dispatch 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[24488]: audit 2024-02-10T08:17:38.771156+0000 mon.a (mon.0) 260 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]': finished 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[24488]: cluster 2024-02-10T08:17:38.771222+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[24488]: audit 2024-02-10T08:17:38.771366+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[17385]: audit 2024-02-10T08:17:38.763135+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.67:0/3958496578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]: dispatch 2024-02-10T08:17:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[17385]: audit 2024-02-10T08:17:38.763769+0000 mon.a (mon.0) 259 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]: dispatch 2024-02-10T08:17:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[17385]: audit 2024-02-10T08:17:38.771156+0000 mon.a (mon.0) 260 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5233196f-2e05-44ce-a209-d6266a187402"}]': finished 2024-02-10T08:17:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[17385]: cluster 2024-02-10T08:17:38.771222+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-10T08:17:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:39 smithi067 bash[17385]: audit 2024-02-10T08:17:38.771366+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:17:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:40 smithi134 bash[18722]: cluster 2024-02-10T08:17:38.838536+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:40 smithi134 bash[18722]: audit 2024-02-10T08:17:39.471461+0000 mon.a (mon.0) 263 : audit [DBG] from='client.? 172.21.15.67:0/1331967640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:17:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:40 smithi067 bash[24488]: cluster 2024-02-10T08:17:38.838536+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:40 smithi067 bash[24488]: audit 2024-02-10T08:17:39.471461+0000 mon.a (mon.0) 263 : audit [DBG] from='client.? 172.21.15.67:0/1331967640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:17:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:40 smithi067 bash[17385]: cluster 2024-02-10T08:17:38.838536+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:40 smithi067 bash[17385]: audit 2024-02-10T08:17:39.471461+0000 mon.a (mon.0) 263 : audit [DBG] from='client.? 172.21.15.67:0/1331967640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:17:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:42 smithi134 bash[18722]: cluster 2024-02-10T08:17:40.839053+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:42 smithi067 bash[24488]: cluster 2024-02-10T08:17:40.839053+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:42 smithi067 bash[17385]: cluster 2024-02-10T08:17:40.839053+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v88: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:44 smithi134 bash[18722]: cluster 2024-02-10T08:17:42.839424+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:44 smithi067 bash[17385]: cluster 2024-02-10T08:17:42.839424+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:44 smithi067 bash[24488]: cluster 2024-02-10T08:17:42.839424+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:46 smithi067 bash[17385]: cluster 2024-02-10T08:17:44.839876+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:46 smithi067 bash[24488]: cluster 2024-02-10T08:17:44.839876+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:46 smithi134 bash[18722]: cluster 2024-02-10T08:17:44.839876+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:48.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:48 smithi067 bash[24488]: cluster 2024-02-10T08:17:46.840317+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:48 smithi067 bash[17385]: cluster 2024-02-10T08:17:46.840317+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:48 smithi134 bash[18722]: cluster 2024-02-10T08:17:46.840317+0000 mgr.y (mgr.14146) 116 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:50 smithi067 bash[24488]: cluster 2024-02-10T08:17:48.840720+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:50 smithi067 bash[17385]: cluster 2024-02-10T08:17:48.840720+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:50 smithi134 bash[18722]: cluster 2024-02-10T08:17:48.840720+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:52 smithi067 bash[17385]: cluster 2024-02-10T08:17:50.841166+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:52 smithi067 bash[17385]: audit 2024-02-10T08:17:51.339362+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T08:17:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:52 smithi067 bash[17385]: audit 2024-02-10T08:17:51.340775+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:52 smithi067 bash[24488]: cluster 2024-02-10T08:17:50.841166+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:52 smithi067 bash[24488]: audit 2024-02-10T08:17:51.339362+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T08:17:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:52 smithi067 bash[24488]: audit 2024-02-10T08:17:51.340775+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:52.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:52 smithi134 bash[18722]: cluster 2024-02-10T08:17:50.841166+0000 mgr.y (mgr.14146) 118 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:52.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:52 smithi134 bash[18722]: audit 2024-02-10T08:17:51.339362+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T08:17:52.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:52 smithi134 bash[18722]: audit 2024-02-10T08:17:51.340775+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:17:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:53 smithi067 bash[17385]: cephadm 2024-02-10T08:17:51.341984+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Deploying daemon osd.1 on smithi067 2024-02-10T08:17:53.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:53 smithi067 bash[24488]: cephadm 2024-02-10T08:17:51.341984+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Deploying daemon osd.1 on smithi067 2024-02-10T08:17:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:53 smithi134 bash[18722]: cephadm 2024-02-10T08:17:51.341984+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Deploying daemon osd.1 on smithi067 2024-02-10T08:17:54.359 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:54 smithi067 bash[17385]: cluster 2024-02-10T08:17:52.841733+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:54.360 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:54 smithi067 bash[24488]: cluster 2024-02-10T08:17:52.841733+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:54 smithi134 bash[18722]: cluster 2024-02-10T08:17:52.841733+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:56 smithi067 bash[17385]: cluster 2024-02-10T08:17:54.842120+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:56 smithi067 bash[17385]: audit 2024-02-10T08:17:55.965023+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:56 smithi067 bash[17385]: audit 2024-02-10T08:17:56.001879+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:56 smithi067 bash[24488]: cluster 2024-02-10T08:17:54.842120+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:56 smithi067 bash[24488]: audit 2024-02-10T08:17:55.965023+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:56 smithi067 bash[24488]: audit 2024-02-10T08:17:56.001879+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:56 smithi134 bash[18722]: cluster 2024-02-10T08:17:54.842120+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:56 smithi134 bash[18722]: audit 2024-02-10T08:17:55.965023+0000 mon.a (mon.0) 266 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:17:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:56 smithi134 bash[18722]: audit 2024-02-10T08:17:56.001879+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:17:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:17:58 smithi134 bash[18722]: cluster 2024-02-10T08:17:56.842641+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:17:58 smithi067 bash[17385]: cluster 2024-02-10T08:17:56.842641+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:17:58.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:17:58 smithi067 bash[24488]: cluster 2024-02-10T08:17:56.842641+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:00.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:00 smithi134 bash[18722]: cluster 2024-02-10T08:17:58.843202+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:00 smithi067 bash[17385]: cluster 2024-02-10T08:17:58.843202+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:00 smithi067 bash[24488]: cluster 2024-02-10T08:17:58.843202+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:02 smithi134 bash[18722]: cluster 2024-02-10T08:18:00.843646+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:02 smithi134 bash[18722]: audit 2024-02-10T08:18:01.132432+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:02 smithi134 bash[18722]: audit 2024-02-10T08:18:01.133766+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:02 smithi134 bash[18722]: audit 2024-02-10T08:18:01.134936+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:02 smithi134 bash[18722]: audit 2024-02-10T08:18:01.140166+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[17385]: cluster 2024-02-10T08:18:00.843646+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[17385]: audit 2024-02-10T08:18:01.132432+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[17385]: audit 2024-02-10T08:18:01.133766+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[17385]: audit 2024-02-10T08:18:01.134936+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[17385]: audit 2024-02-10T08:18:01.140166+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[24488]: cluster 2024-02-10T08:18:00.843646+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[24488]: audit 2024-02-10T08:18:01.132432+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[24488]: audit 2024-02-10T08:18:01.133766+0000 mon.a (mon.0) 269 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[24488]: audit 2024-02-10T08:18:01.134936+0000 mon.a (mon.0) 270 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:02 smithi067 bash[24488]: audit 2024-02-10T08:18:01.140166+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:03 smithi134 bash[18722]: audit 2024-02-10T08:18:02.647932+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T08:18:03.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:03 smithi067 bash[17385]: audit 2024-02-10T08:18:02.647932+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T08:18:03.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:03 smithi067 bash[24488]: audit 2024-02-10T08:18:02.647932+0000 mon.a (mon.0) 272 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[17385]: cluster 2024-02-10T08:18:02.844154+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[17385]: audit 2024-02-10T08:18:03.033509+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[17385]: cluster 2024-02-10T08:18:03.033632+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[17385]: audit 2024-02-10T08:18:03.033825+0000 mon.a (mon.0) 275 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[17385]: audit 2024-02-10T08:18:03.034374+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[24488]: cluster 2024-02-10T08:18:02.844154+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[24488]: audit 2024-02-10T08:18:03.033509+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T08:18:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[24488]: cluster 2024-02-10T08:18:03.033632+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-10T08:18:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[24488]: audit 2024-02-10T08:18:03.033825+0000 mon.a (mon.0) 275 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:04 smithi067 bash[24488]: audit 2024-02-10T08:18:03.034374+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:04 smithi134 bash[18722]: cluster 2024-02-10T08:18:02.844154+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:04 smithi134 bash[18722]: audit 2024-02-10T08:18:03.033509+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T08:18:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:04 smithi134 bash[18722]: cluster 2024-02-10T08:18:03.033632+0000 mon.a (mon.0) 274 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-10T08:18:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:04 smithi134 bash[18722]: audit 2024-02-10T08:18:03.033825+0000 mon.a (mon.0) 275 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:04 smithi134 bash[18722]: audit 2024-02-10T08:18:03.034374+0000 mon.a (mon.0) 276 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:04.626 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.618+0000 7f6d377fe700 1 -- 172.21.15.67:0/697927284 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6d400591c0 con 0x7f6d2c061250 2024-02-10T08:18:04.626 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 1 on host 'smithi067' 2024-02-10T08:18:04.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f6d2c061250 msgr2=0x7f6d2c063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:04.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f6d2c061250 0x7f6d2c063710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6d30006fd0 tx=0x7f6d30009060).stop 2024-02-10T08:18:04.629 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 msgr2=0x7f6d4010e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d4010e330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6d3c00eca0 tx=0x7f6d3c00b040).stop 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 shutdown_connections 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f6d2c061250 0x7f6d2c063710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6d40061770 0x7f6d4010e330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d400fb530 0x7f6d40115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 --2- 172.21.15.67:0/697927284 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6d400f8a80 0x7f6d4010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 >> 172.21.15.67:0/697927284 conn(0x7f6d400f2230 msgr2=0x7f6d400f6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 shutdown_connections 2024-02-10T08:18:04.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:04.622+0000 7f6d4836b700 1 -- 172.21.15.67:0/697927284 wait complete. 2024-02-10T08:18:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[17385]: audit 2024-02-10T08:18:04.036959+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:18:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[17385]: cluster 2024-02-10T08:18:04.037157+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-10T08:18:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[17385]: audit 2024-02-10T08:18:04.037938+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[17385]: audit 2024-02-10T08:18:04.044792+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[17385]: audit 2024-02-10T08:18:04.615994+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[17385]: audit 2024-02-10T08:18:04.623291+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[24488]: audit 2024-02-10T08:18:04.036959+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:18:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[24488]: cluster 2024-02-10T08:18:04.037157+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-10T08:18:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[24488]: audit 2024-02-10T08:18:04.037938+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[24488]: audit 2024-02-10T08:18:04.044792+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[24488]: audit 2024-02-10T08:18:04.615994+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:05 smithi067 bash[24488]: audit 2024-02-10T08:18:04.623291+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:05.416 DEBUG:teuthology.orchestra.run.smithi067:osd.1> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.1.service 2024-02-10T08:18:05.420 INFO:tasks.cephadm:Deploying osd.2 on smithi067 with /dev/vg_nvme/lv_2... 2024-02-10T08:18:05.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-10T08:18:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:05 smithi134 bash[18722]: audit 2024-02-10T08:18:04.036959+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:18:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:05 smithi134 bash[18722]: cluster 2024-02-10T08:18:04.037157+0000 mon.a (mon.0) 278 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-10T08:18:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:05 smithi134 bash[18722]: audit 2024-02-10T08:18:04.037938+0000 mon.a (mon.0) 279 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:05 smithi134 bash[18722]: audit 2024-02-10T08:18:04.044792+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:05 smithi134 bash[18722]: audit 2024-02-10T08:18:04.615994+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:05 smithi134 bash[18722]: audit 2024-02-10T08:18:04.623291+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:06.181 INFO:journalctl@ceph.osd.1.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:18:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[17385]: cluster 2024-02-10T08:18:03.675387+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:18:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[17385]: cluster 2024-02-10T08:18:03.675492+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:18:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[17385]: cluster 2024-02-10T08:18:04.844597+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[17385]: cluster 2024-02-10T08:18:05.046054+0000 mon.a (mon.0) 283 : cluster [INF] osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] boot 2024-02-10T08:18:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[17385]: cluster 2024-02-10T08:18:05.046101+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-10T08:18:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[17385]: audit 2024-02-10T08:18:05.046841+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[24488]: cluster 2024-02-10T08:18:03.675387+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:18:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[24488]: cluster 2024-02-10T08:18:03.675492+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:18:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[24488]: cluster 2024-02-10T08:18:04.844597+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[24488]: cluster 2024-02-10T08:18:05.046054+0000 mon.a (mon.0) 283 : cluster [INF] osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] boot 2024-02-10T08:18:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[24488]: cluster 2024-02-10T08:18:05.046101+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-10T08:18:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:06 smithi067 bash[24488]: audit 2024-02-10T08:18:05.046841+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:06 smithi134 bash[18722]: cluster 2024-02-10T08:18:03.675387+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:18:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:06 smithi134 bash[18722]: cluster 2024-02-10T08:18:03.675492+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:18:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:06 smithi134 bash[18722]: cluster 2024-02-10T08:18:04.844597+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-10T08:18:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:06 smithi134 bash[18722]: cluster 2024-02-10T08:18:05.046054+0000 mon.a (mon.0) 283 : cluster [INF] osd.1 [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] boot 2024-02-10T08:18:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:06 smithi134 bash[18722]: cluster 2024-02-10T08:18:05.046101+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-10T08:18:06.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:06 smithi134 bash[18722]: audit 2024-02-10T08:18:05.046841+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:18:07.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:07 smithi067 bash[17385]: cluster 2024-02-10T08:18:06.630106+0000 mon.a (mon.0) 286 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-10T08:18:07.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:07 smithi067 bash[17385]: cluster 2024-02-10T08:18:06.845052+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:07.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:07 smithi067 bash[24488]: cluster 2024-02-10T08:18:06.630106+0000 mon.a (mon.0) 286 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-10T08:18:07.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:07 smithi067 bash[24488]: cluster 2024-02-10T08:18:06.845052+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:07 smithi134 bash[18722]: cluster 2024-02-10T08:18:06.630106+0000 mon.a (mon.0) 286 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-10T08:18:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:07 smithi134 bash[18722]: cluster 2024-02-10T08:18:06.845052+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-10T08:18:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-10T08:18:08.970 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-02-10T08:18:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-02-10T08:18:08.970 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0181548 s, 578 MB/s 2024-02-10T08:18:08.970 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-02-10T08:18:09.661 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-02-10T08:18:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:09 smithi067 bash[17385]: cluster 2024-02-10T08:18:08.845554+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:09 smithi067 bash[24488]: cluster 2024-02-10T08:18:08.845554+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:09 smithi134 bash[18722]: cluster 2024-02-10T08:18:08.845554+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:12.059 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:11 smithi067 bash[24488]: cluster 2024-02-10T08:18:10.846105+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:12.059 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:11 smithi067 bash[17385]: cluster 2024-02-10T08:18:10.846105+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:11 smithi134 bash[18722]: cluster 2024-02-10T08:18:10.846105+0000 mgr.y (mgr.14146) 129 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 -- 172.21.15.67:0/1006525219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f8430 msgr2=0x7f46ec0f8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:14.146 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/1006525219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f8430 0x7f46ec0f8e60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f46e0004870 tx=0x7f46e0009cb0).stop 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 -- 172.21.15.67:0/1006525219 shutdown_connections 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/1006525219 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f9540 0x7f46ec1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/1006525219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec0f7b20 0x7f46ec0f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/1006525219 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f8430 0x7f46ec0f8e60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 -- 172.21.15.67:0/1006525219 >> 172.21.15.67:0/1006525219 conn(0x7f46ec0f3340 msgr2=0x7f46ec0f5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 -- 172.21.15.67:0/1006525219 shutdown_connections 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.142+0000 7f46f1a9d700 1 -- 172.21.15.67:0/1006525219 wait complete. 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 Processor -- start 2024-02-10T08:18:14.147 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 -- start start 2024-02-10T08:18:14.148 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec0f7b20 0x7f46ec107e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f8430 0x7f46ec108370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 0x7f46ec10f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec10f8f0 con 0x7f46ec0f9540 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec10fa30 con 0x7f46ec0f7b20 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec10fb70 con 0x7f46ec0f8430 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f8430 0x7f46ec108370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46ebfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 0x7f46ec10f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46ebfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 0x7f46ec10f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33622/0 (socket says 172.21.15.67:33622) 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46ebfff700 1 -- 172.21.15.67:0/2395958661 learned_addr learned my addr 172.21.15.67:0/2395958661 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eb7fe700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec0f7b20 0x7f46ec107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 -- 172.21.15.67:0/2395958661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec0f7b20 msgr2=0x7f46ec107e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:14.149 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec0f7b20 0x7f46ec107e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 -- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 msgr2=0x7f46ec10f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 0x7f46ec10f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 -- 172.21.15.67:0/2395958661 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0005040 con 0x7f46ec0f8430 2024-02-10T08:18:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46ebfff700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 0x7f46ec10f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:18:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46eaffd700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f8430 0x7f46ec108370 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f46e0004710 tx=0x7f46e0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:18:14.150 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e000e030 con 0x7f46ec0f8430 2024-02-10T08:18:14.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ec10fdf0 con 0x7f46ec0f8430 2024-02-10T08:18:14.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec110220 con 0x7f46ec0f8430 2024-02-10T08:18:14.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f46e00096d0 con 0x7f46ec0f8430 2024-02-10T08:18:14.151 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.146+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0018070 con 0x7f46ec0f8430 2024-02-10T08:18:14.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.150+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d8004f50 con 0x7f46ec0f8430 2024-02-10T08:18:14.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.150+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f46e0009240 con 0x7f46ec0f8430 2024-02-10T08:18:14.152 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.150+0000 7f46e8f74700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f46d4061250 0x7f46d4063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:14.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.150+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2142+0+0 (secure 0 0 0) 0x7f46e007cb60 con 0x7f46ec0f8430 2024-02-10T08:18:14.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.150+0000 7f46eb7fe700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f46d4061250 0x7f46d4063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:14.153 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.150+0000 7f46eb7fe700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f46d4061250 0x7f46d4063710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f46dc009fd0 tx=0x7f46dc009d50).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:18:14.157 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.154+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f46e004dd60 con 0x7f46ec0f8430 2024-02-10T08:18:14.234 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:14 smithi067 bash[17385]: cluster 2024-02-10T08:18:12.846632+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:14.234 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:14 smithi067 bash[24488]: cluster 2024-02-10T08:18:12.846632+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:14.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:14 smithi134 bash[18722]: cluster 2024-02-10T08:18:12.846632+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:14.355 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:14.354+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d8000bf0 con 0x7f46d4061250 2024-02-10T08:18:15.364 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:15 smithi067 bash[17385]: audit 2024-02-10T08:18:14.362455+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:18:15.365 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:15 smithi067 bash[17385]: audit 2024-02-10T08:18:14.366089+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:18:15.365 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:15 smithi067 bash[17385]: audit 2024-02-10T08:18:14.367172+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:15.365 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:15 smithi067 bash[24488]: audit 2024-02-10T08:18:14.362455+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:18:15.365 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:15 smithi067 bash[24488]: audit 2024-02-10T08:18:14.366089+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:18:15.365 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:15 smithi067 bash[24488]: audit 2024-02-10T08:18:14.367172+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:15 smithi134 bash[18722]: audit 2024-02-10T08:18:14.362455+0000 mon.a (mon.0) 287 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:18:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:15 smithi134 bash[18722]: audit 2024-02-10T08:18:14.366089+0000 mon.a (mon.0) 288 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:18:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:15 smithi134 bash[18722]: audit 2024-02-10T08:18:14.367172+0000 mon.a (mon.0) 289 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:16 smithi067 bash[17385]: audit 2024-02-10T08:18:14.356213+0000 mgr.y (mgr.14146) 131 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:18:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:16 smithi067 bash[17385]: cluster 2024-02-10T08:18:14.846991+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:16.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:16 smithi067 bash[24488]: audit 2024-02-10T08:18:14.356213+0000 mgr.y (mgr.14146) 131 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:18:16.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:16 smithi067 bash[24488]: cluster 2024-02-10T08:18:14.846991+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:16 smithi134 bash[18722]: audit 2024-02-10T08:18:14.356213+0000 mgr.y (mgr.14146) 131 : audit [DBG] from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:18:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:16 smithi134 bash[18722]: cluster 2024-02-10T08:18:14.846991+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: cephadm 2024-02-10T08:18:16.138735+0000 mgr.y (mgr.14146) 133 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:18:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.146817+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.148289+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.149419+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: cephadm 2024-02-10T08:18:16.150228+0000 mgr.y (mgr.14146) 134 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 8120M 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.154967+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.156343+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.157617+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: audit 2024-02-10T08:18:16.164357+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:17 smithi134 bash[18722]: cluster 2024-02-10T08:18:16.847371+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: cephadm 2024-02-10T08:18:16.138735+0000 mgr.y (mgr.14146) 133 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:18:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.146817+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.148289+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:18:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.149419+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: cephadm 2024-02-10T08:18:16.150228+0000 mgr.y (mgr.14146) 134 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 8120M 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.154967+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.156343+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.157617+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: audit 2024-02-10T08:18:16.164357+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[17385]: cluster 2024-02-10T08:18:16.847371+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: cephadm 2024-02-10T08:18:16.138735+0000 mgr.y (mgr.14146) 133 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.146817+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.148289+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.149419+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: cephadm 2024-02-10T08:18:16.150228+0000 mgr.y (mgr.14146) 134 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 8120M 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.154967+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.156343+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:17.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.157617+0000 mon.a (mon.0) 295 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:17.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: audit 2024-02-10T08:18:16.164357+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:17.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:17 smithi067 bash[24488]: cluster 2024-02-10T08:18:16.847371+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:20.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:19 smithi134 bash[18722]: cluster 2024-02-10T08:18:18.847823+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:20.315 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:19 smithi067 bash[17385]: cluster 2024-02-10T08:18:18.847823+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:20.316 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:19 smithi067 bash[24488]: cluster 2024-02-10T08:18:18.847823+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v111: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:22.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:21 smithi134 bash[18722]: cluster 2024-02-10T08:18:20.848233+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:22.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:21 smithi067 bash[24488]: cluster 2024-02-10T08:18:20.848233+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:21 smithi067 bash[17385]: cluster 2024-02-10T08:18:20.848233+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:23.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:23 smithi134 bash[18722]: audit 2024-02-10T08:18:22.843288+0000 mon.c (mon.2) 9 : audit [INF] from='client.? 172.21.15.67:0/852689743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]: dispatch 2024-02-10T08:18:23.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:23 smithi134 bash[18722]: audit 2024-02-10T08:18:22.844111+0000 mon.a (mon.0) 297 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]: dispatch 2024-02-10T08:18:23.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:23 smithi134 bash[18722]: audit 2024-02-10T08:18:22.853862+0000 mon.a (mon.0) 298 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]': finished 2024-02-10T08:18:23.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:23 smithi134 bash[18722]: cluster 2024-02-10T08:18:22.853957+0000 mon.a (mon.0) 299 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-10T08:18:23.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:23 smithi134 bash[18722]: audit 2024-02-10T08:18:22.854851+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[17385]: audit 2024-02-10T08:18:22.843288+0000 mon.c (mon.2) 9 : audit [INF] from='client.? 172.21.15.67:0/852689743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]: dispatch 2024-02-10T08:18:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[17385]: audit 2024-02-10T08:18:22.844111+0000 mon.a (mon.0) 297 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]: dispatch 2024-02-10T08:18:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[17385]: audit 2024-02-10T08:18:22.853862+0000 mon.a (mon.0) 298 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]': finished 2024-02-10T08:18:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[17385]: cluster 2024-02-10T08:18:22.853957+0000 mon.a (mon.0) 299 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-10T08:18:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[17385]: audit 2024-02-10T08:18:22.854851+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[24488]: audit 2024-02-10T08:18:22.843288+0000 mon.c (mon.2) 9 : audit [INF] from='client.? 172.21.15.67:0/852689743' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]: dispatch 2024-02-10T08:18:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[24488]: audit 2024-02-10T08:18:22.844111+0000 mon.a (mon.0) 297 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]: dispatch 2024-02-10T08:18:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[24488]: audit 2024-02-10T08:18:22.853862+0000 mon.a (mon.0) 298 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b812fb6-4df3-47ad-a32d-5d84545750b2"}]': finished 2024-02-10T08:18:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[24488]: cluster 2024-02-10T08:18:22.853957+0000 mon.a (mon.0) 299 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-10T08:18:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:23 smithi067 bash[24488]: audit 2024-02-10T08:18:22.854851+0000 mon.a (mon.0) 300 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:24 smithi134 bash[18722]: cluster 2024-02-10T08:18:22.848653+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:24 smithi134 bash[18722]: audit 2024-02-10T08:18:23.609680+0000 mon.a (mon.0) 301 : audit [DBG] from='client.? 172.21.15.67:0/218755037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:18:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:24 smithi067 bash[17385]: cluster 2024-02-10T08:18:22.848653+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:24 smithi067 bash[17385]: audit 2024-02-10T08:18:23.609680+0000 mon.a (mon.0) 301 : audit [DBG] from='client.? 172.21.15.67:0/218755037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:18:24.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:24 smithi067 bash[24488]: cluster 2024-02-10T08:18:22.848653+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v113: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:24.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:24 smithi067 bash[24488]: audit 2024-02-10T08:18:23.609680+0000 mon.a (mon.0) 301 : audit [DBG] from='client.? 172.21.15.67:0/218755037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:18:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:26 smithi067 bash[17385]: cluster 2024-02-10T08:18:24.849206+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:26 smithi067 bash[24488]: cluster 2024-02-10T08:18:24.849206+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:26 smithi134 bash[18722]: cluster 2024-02-10T08:18:24.849206+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:28.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:28 smithi067 bash[17385]: cluster 2024-02-10T08:18:26.849646+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:28.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:28 smithi067 bash[24488]: cluster 2024-02-10T08:18:26.849646+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:28 smithi134 bash[18722]: cluster 2024-02-10T08:18:26.849646+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v116: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:30 smithi067 bash[17385]: cluster 2024-02-10T08:18:28.850185+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:30 smithi067 bash[24488]: cluster 2024-02-10T08:18:28.850185+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:30 smithi134 bash[18722]: cluster 2024-02-10T08:18:28.850185+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v117: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:32 smithi067 bash[17385]: cluster 2024-02-10T08:18:30.850794+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:32 smithi067 bash[24488]: cluster 2024-02-10T08:18:30.850794+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:32 smithi134 bash[18722]: cluster 2024-02-10T08:18:30.850794+0000 mgr.y (mgr.14146) 142 : cluster [DBG] pgmap v118: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:34 smithi067 bash[17385]: cluster 2024-02-10T08:18:32.851356+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:34 smithi067 bash[24488]: cluster 2024-02-10T08:18:32.851356+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:34 smithi134 bash[18722]: cluster 2024-02-10T08:18:32.851356+0000 mgr.y (mgr.14146) 143 : cluster [DBG] pgmap v119: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:36 smithi067 bash[17385]: cluster 2024-02-10T08:18:34.851857+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:36 smithi067 bash[17385]: audit 2024-02-10T08:18:35.560754+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T08:18:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:36 smithi067 bash[17385]: audit 2024-02-10T08:18:35.562365+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:36.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:36 smithi067 bash[24488]: cluster 2024-02-10T08:18:34.851857+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:36 smithi067 bash[24488]: audit 2024-02-10T08:18:35.560754+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T08:18:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:36 smithi067 bash[24488]: audit 2024-02-10T08:18:35.562365+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:36.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:36 smithi134 bash[18722]: cluster 2024-02-10T08:18:34.851857+0000 mgr.y (mgr.14146) 144 : cluster [DBG] pgmap v120: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:36.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:36 smithi134 bash[18722]: audit 2024-02-10T08:18:35.560754+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T08:18:36.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:36 smithi134 bash[18722]: audit 2024-02-10T08:18:35.562365+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:37.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:37 smithi067 bash[17385]: cephadm 2024-02-10T08:18:35.563802+0000 mgr.y (mgr.14146) 145 : cephadm [INF] Deploying daemon osd.2 on smithi067 2024-02-10T08:18:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:37 smithi067 bash[24488]: cephadm 2024-02-10T08:18:35.563802+0000 mgr.y (mgr.14146) 145 : cephadm [INF] Deploying daemon osd.2 on smithi067 2024-02-10T08:18:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:37 smithi134 bash[18722]: cephadm 2024-02-10T08:18:35.563802+0000 mgr.y (mgr.14146) 145 : cephadm [INF] Deploying daemon osd.2 on smithi067 2024-02-10T08:18:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:38 smithi067 bash[17385]: cluster 2024-02-10T08:18:36.852466+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:38.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:38 smithi067 bash[24488]: cluster 2024-02-10T08:18:36.852466+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:38 smithi134 bash[18722]: cluster 2024-02-10T08:18:36.852466+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v121: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:40 smithi067 bash[24488]: cluster 2024-02-10T08:18:38.852910+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:40 smithi067 bash[24488]: audit 2024-02-10T08:18:39.988866+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:40 smithi067 bash[24488]: audit 2024-02-10T08:18:39.997839+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:40 smithi067 bash[17385]: cluster 2024-02-10T08:18:38.852910+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:40 smithi067 bash[17385]: audit 2024-02-10T08:18:39.988866+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:40 smithi067 bash[17385]: audit 2024-02-10T08:18:39.997839+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:40 smithi134 bash[18722]: cluster 2024-02-10T08:18:38.852910+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v122: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:40 smithi134 bash[18722]: audit 2024-02-10T08:18:39.988866+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:40 smithi134 bash[18722]: audit 2024-02-10T08:18:39.997839+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:41 smithi067 bash[24488]: cluster 2024-02-10T08:18:40.853427+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:41.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:41 smithi067 bash[17385]: cluster 2024-02-10T08:18:40.853427+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:41 smithi134 bash[18722]: cluster 2024-02-10T08:18:40.853427+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v123: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:44 smithi067 bash[17385]: cluster 2024-02-10T08:18:42.854007+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:44 smithi067 bash[24488]: cluster 2024-02-10T08:18:42.854007+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:44 smithi134 bash[18722]: cluster 2024-02-10T08:18:42.854007+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v124: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[17385]: cluster 2024-02-10T08:18:44.854608+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[17385]: audit 2024-02-10T08:18:45.360842+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[17385]: audit 2024-02-10T08:18:45.650874+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[17385]: audit 2024-02-10T08:18:45.652261+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[17385]: audit 2024-02-10T08:18:45.660907+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[24488]: cluster 2024-02-10T08:18:44.854608+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:46.253 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[24488]: audit 2024-02-10T08:18:45.360842+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:46.254 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[24488]: audit 2024-02-10T08:18:45.650874+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:46.254 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[24488]: audit 2024-02-10T08:18:45.652261+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:46.254 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:46 smithi067 bash[24488]: audit 2024-02-10T08:18:45.660907+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:46 smithi134 bash[18722]: cluster 2024-02-10T08:18:44.854608+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v125: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:46 smithi134 bash[18722]: audit 2024-02-10T08:18:45.360842+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:46 smithi134 bash[18722]: audit 2024-02-10T08:18:45.650874+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:46 smithi134 bash[18722]: audit 2024-02-10T08:18:45.652261+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:18:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:46 smithi134 bash[18722]: audit 2024-02-10T08:18:45.660907+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:47 smithi067 bash[17385]: audit 2024-02-10T08:18:46.828991+0000 mon.c (mon.2) 10 : audit [INF] from='osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T08:18:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:47 smithi067 bash[17385]: audit 2024-02-10T08:18:46.829617+0000 mon.a (mon.0) 310 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T08:18:47.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:47 smithi067 bash[24488]: audit 2024-02-10T08:18:46.828991+0000 mon.c (mon.2) 10 : audit [INF] from='osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T08:18:47.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:47 smithi067 bash[24488]: audit 2024-02-10T08:18:46.829617+0000 mon.a (mon.0) 310 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T08:18:47.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:47 smithi134 bash[18722]: audit 2024-02-10T08:18:46.828991+0000 mon.c (mon.2) 10 : audit [INF] from='osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T08:18:47.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:47 smithi134 bash[18722]: audit 2024-02-10T08:18:46.829617+0000 mon.a (mon.0) 310 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T08:18:48.050 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[24488]: cluster 2024-02-10T08:18:46.855106+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:48.051 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[24488]: audit 2024-02-10T08:18:47.040118+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T08:18:48.343 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[24488]: cluster 2024-02-10T08:18:47.040186+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-10T08:18:48.343 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[24488]: audit 2024-02-10T08:18:47.040339+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[24488]: audit 2024-02-10T08:18:47.041241+0000 mon.c (mon.2) 11 : audit [INF] from='osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[24488]: audit 2024-02-10T08:18:47.041669+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[17385]: cluster 2024-02-10T08:18:46.855106+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[17385]: audit 2024-02-10T08:18:47.040118+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[17385]: cluster 2024-02-10T08:18:47.040186+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[17385]: audit 2024-02-10T08:18:47.040339+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[17385]: audit 2024-02-10T08:18:47.041241+0000 mon.c (mon.2) 11 : audit [INF] from='osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:48.344 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:48 smithi067 bash[17385]: audit 2024-02-10T08:18:47.041669+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:48 smithi134 bash[18722]: cluster 2024-02-10T08:18:46.855106+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v126: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:48 smithi134 bash[18722]: audit 2024-02-10T08:18:47.040118+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T08:18:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:48 smithi134 bash[18722]: cluster 2024-02-10T08:18:47.040186+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-10T08:18:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:48 smithi134 bash[18722]: audit 2024-02-10T08:18:47.040339+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:48.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:48 smithi134 bash[18722]: audit 2024-02-10T08:18:47.041241+0000 mon.c (mon.2) 11 : audit [INF] from='osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:48.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:48 smithi134 bash[18722]: audit 2024-02-10T08:18:47.041669+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:18:48.820 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.817+0000 7f46e8f74700 1 -- 172.21.15.67:0/2395958661 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f46d8000bf0 con 0x7f46d4061250 2024-02-10T08:18:48.820 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 2 on host 'smithi067' 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f46d4061250 msgr2=0x7f46d4063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f46d4061250 0x7f46d4063710 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f46dc009fd0 tx=0x7f46dc009d50).stop 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f8430 msgr2=0x7f46ec108370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f8430 0x7f46ec108370 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f46e0004710 tx=0x7f46e0009cb0).stop 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 shutdown_connections 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f46d4061250 0x7f46d4063710 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f46ec0f9540 0x7f46ec10f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46ec0f7b20 0x7f46ec107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 --2- 172.21.15.67:0/2395958661 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f46ec0f8430 0x7f46ec108370 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 >> 172.21.15.67:0/2395958661 conn(0x7f46ec0f3340 msgr2=0x7f46ec0f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 shutdown_connections 2024-02-10T08:18:48.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:48.821+0000 7f46f1a9d700 1 -- 172.21.15.67:0/2395958661 wait complete. 2024-02-10T08:18:49.548 DEBUG:teuthology.orchestra.run.smithi067:osd.2> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.2.service 2024-02-10T08:18:49.550 INFO:tasks.cephadm:Deploying osd.3 on smithi067 with /dev/vg_nvme/lv_1... 2024-02-10T08:18:49.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-10T08:18:49.653 INFO:journalctl@ceph.osd.2.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[17385]: audit 2024-02-10T08:18:48.044869+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[17385]: cluster 2024-02-10T08:18:48.044978+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[17385]: audit 2024-02-10T08:18:48.045547+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[17385]: audit 2024-02-10T08:18:48.052961+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[17385]: audit 2024-02-10T08:18:48.818796+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[17385]: audit 2024-02-10T08:18:48.836124+0000 mon.a (mon.0) 320 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[24488]: audit 2024-02-10T08:18:48.044869+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:18:49.792 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[24488]: cluster 2024-02-10T08:18:48.044978+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-10T08:18:49.793 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[24488]: audit 2024-02-10T08:18:48.045547+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:49.793 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[24488]: audit 2024-02-10T08:18:48.052961+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:49.793 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[24488]: audit 2024-02-10T08:18:48.818796+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:49.793 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:49 smithi067 bash[24488]: audit 2024-02-10T08:18:48.836124+0000 mon.a (mon.0) 320 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:49 smithi134 bash[18722]: audit 2024-02-10T08:18:48.044869+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:18:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:49 smithi134 bash[18722]: cluster 2024-02-10T08:18:48.044978+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-10T08:18:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:49 smithi134 bash[18722]: audit 2024-02-10T08:18:48.045547+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:49 smithi134 bash[18722]: audit 2024-02-10T08:18:48.052961+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:49 smithi134 bash[18722]: audit 2024-02-10T08:18:48.818796+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:18:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:49 smithi134 bash[18722]: audit 2024-02-10T08:18:48.836124+0000 mon.a (mon.0) 320 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: cluster 2024-02-10T08:18:47.802811+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: cluster 2024-02-10T08:18:47.802900+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: cluster 2024-02-10T08:18:48.855567+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: audit 2024-02-10T08:18:49.050089+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: cluster 2024-02-10T08:18:49.056963+0000 mon.a (mon.0) 322 : cluster [INF] osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] boot 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: cluster 2024-02-10T08:18:49.057008+0000 mon.a (mon.0) 323 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: audit 2024-02-10T08:18:49.057485+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: audit 2024-02-10T08:18:49.057822+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:18:50.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: audit 2024-02-10T08:18:50.057297+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-10T08:18:50.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: cluster 2024-02-10T08:18:50.057385+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-10T08:18:50.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[17385]: audit 2024-02-10T08:18:50.058232+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: cluster 2024-02-10T08:18:47.802811+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: cluster 2024-02-10T08:18:47.802900+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: cluster 2024-02-10T08:18:48.855567+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: audit 2024-02-10T08:18:49.050089+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: cluster 2024-02-10T08:18:49.056963+0000 mon.a (mon.0) 322 : cluster [INF] osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] boot 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: cluster 2024-02-10T08:18:49.057008+0000 mon.a (mon.0) 323 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: audit 2024-02-10T08:18:49.057485+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:50.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: audit 2024-02-10T08:18:49.057822+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:18:50.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: audit 2024-02-10T08:18:50.057297+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-10T08:18:50.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: cluster 2024-02-10T08:18:50.057385+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-10T08:18:50.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:50 smithi067 bash[24488]: audit 2024-02-10T08:18:50.058232+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:18:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: cluster 2024-02-10T08:18:47.802811+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:18:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: cluster 2024-02-10T08:18:47.802900+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: cluster 2024-02-10T08:18:48.855567+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v129: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: audit 2024-02-10T08:18:49.050089+0000 mon.a (mon.0) 321 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: cluster 2024-02-10T08:18:49.056963+0000 mon.a (mon.0) 322 : cluster [INF] osd.2 [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] boot 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: cluster 2024-02-10T08:18:49.057008+0000 mon.a (mon.0) 323 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: audit 2024-02-10T08:18:49.057485+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: audit 2024-02-10T08:18:49.057822+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: audit 2024-02-10T08:18:50.057297+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: cluster 2024-02-10T08:18:50.057385+0000 mon.a (mon.0) 327 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-10T08:18:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:50 smithi134 bash[18722]: audit 2024-02-10T08:18:50.058232+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:18:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:52 smithi067 bash[17385]: cluster 2024-02-10T08:18:50.856052+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v132: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:52 smithi067 bash[17385]: audit 2024-02-10T08:18:51.061472+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-10T08:18:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:52 smithi067 bash[17385]: cluster 2024-02-10T08:18:51.061630+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-10T08:18:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:52 smithi067 bash[24488]: cluster 2024-02-10T08:18:50.856052+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v132: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:52 smithi067 bash[24488]: audit 2024-02-10T08:18:51.061472+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-10T08:18:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:52 smithi067 bash[24488]: cluster 2024-02-10T08:18:51.061630+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-10T08:18:52.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:52 smithi134 bash[18722]: cluster 2024-02-10T08:18:50.856052+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v132: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:52.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:52 smithi134 bash[18722]: audit 2024-02-10T08:18:51.061472+0000 mon.a (mon.0) 329 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-10T08:18:52.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:52 smithi134 bash[18722]: cluster 2024-02-10T08:18:51.061630+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-10T08:18:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-10T08:18:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-10T08:18:53.095 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-02-10T08:18:53.095 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-02-10T08:18:53.095 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0215116 s, 487 MB/s 2024-02-10T08:18:53.096 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-02-10T08:18:53.335 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:53 smithi067 bash[24488]: cluster 2024-02-10T08:18:52.077369+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-10T08:18:53.335 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:53 smithi067 bash[24488]: cluster 2024-02-10T08:18:52.856505+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v135: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:53.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:53 smithi067 bash[17385]: cluster 2024-02-10T08:18:52.077369+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-10T08:18:53.335 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:53 smithi067 bash[17385]: cluster 2024-02-10T08:18:52.856505+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v135: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:53 smithi134 bash[18722]: cluster 2024-02-10T08:18:52.077369+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-10T08:18:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:53 smithi134 bash[18722]: cluster 2024-02-10T08:18:52.856505+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v135: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:53.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-02-10T08:18:56.046 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:55 smithi067 bash[17385]: cluster 2024-02-10T08:18:54.856922+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v136: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:55 smithi134 bash[18722]: cluster 2024-02-10T08:18:54.856922+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v136: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:56 smithi067 bash[24488]: cluster 2024-02-10T08:18:54.856922+0000 mgr.y (mgr.14146) 155 : cluster [DBG] pgmap v136: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:58.309 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:58 smithi067 bash[17385]: cluster 2024-02-10T08:18:56.857372+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:58.310 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:58 smithi067 bash[24488]: cluster 2024-02-10T08:18:56.857372+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:58.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- 172.21.15.67:0/2331311482 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 msgr2=0x7f94a0102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:58.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2331311482 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a0102220 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f9498009c10 tx=0x7f9498004d10).stop 2024-02-10T08:18:58.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- 172.21.15.67:0/2331311482 shutdown_connections 2024-02-10T08:18:58.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2331311482 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00f9d30 0x7f94a00fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:58.467 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2331311482 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a0102220 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2331311482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00fafd0 0x7f94a00fb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- 172.21.15.67:0/2331311482 >> 172.21.15.67:0/2331311482 conn(0x7f94a00f5710 msgr2=0x7f94a00f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:18:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- 172.21.15.67:0/2331311482 shutdown_connections 2024-02-10T08:18:58.468 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- 172.21.15.67:0/2331311482 wait complete. 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 Processor -- start 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- start start 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00f9d30 0x7f94a0109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00fafd0 0x7f94a010a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a01114b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a01119a0 con 0x7f94a00fafd0 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0111ae0 con 0x7f94a00f9d30 2024-02-10T08:18:58.469 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f94a56e4700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0111c20 con 0x7f94a00fb920 2024-02-10T08:18:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a01114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00fafd0 0x7f94a010a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a01114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:40970/0 (socket says 172.21.15.67:40970) 2024-02-10T08:18:58.470 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949effd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00f9d30 0x7f94a0109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 -- 172.21.15.67:0/2325809425 learned_addr learned my addr 172.21.15.67:0/2325809425 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 -- 172.21.15.67:0/2325809425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00f9d30 msgr2=0x7f94a0109ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00f9d30 0x7f94a0109ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 -- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00fafd0 msgr2=0x7f94a010a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00fafd0 0x7f94a010a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 -- 172.21.15.67:0/2325809425 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980097c0 con 0x7f94a00fb920 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949e7fc700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00fafd0 0x7f94a010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:18:58.471 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949effd700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00f9d30 0x7f94a0109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.465+0000 7f949f7fe700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a01114b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f9498009be0 tx=0x7f9498004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949800e070 con 0x7f94a00fb920 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f9498018040 con 0x7f94a00fb920 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0111ea0 con 0x7f94a00fb920 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498017070 con 0x7f94a00fb920 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a01121f0 con 0x7f94a00fb920 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0060070 con 0x7f94a00fb920 2024-02-10T08:18:58.472 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f949801d410 con 0x7f94a00fb920 2024-02-10T08:18:58.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f9487fff700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9488061250 0x7f9488063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:18:58.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2990+0+0 (secure 0 0 0) 0x7f949807d110 con 0x7f94a00fb920 2024-02-10T08:18:58.473 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f949effd700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9488061250 0x7f9488063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:18:58.474 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.469+0000 7f949effd700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9488061250 0x7f9488063710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f94a00fae30 tx=0x7f949400b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:18:58.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.477+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f949804e000 con 0x7f94a00fb920 2024-02-10T08:18:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:58 smithi134 bash[18722]: cluster 2024-02-10T08:18:56.857372+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:18:58.689 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:18:58.685+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a0101000 con 0x7f9488061250 2024-02-10T08:18:59.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:59 smithi067 bash[24488]: audit 2024-02-10T08:18:58.695933+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:18:59.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:59 smithi067 bash[24488]: audit 2024-02-10T08:18:58.698803+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:18:59.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:18:59 smithi067 bash[24488]: audit 2024-02-10T08:18:58.699656+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:59.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:59 smithi067 bash[17385]: audit 2024-02-10T08:18:58.695933+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:18:59.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:59 smithi067 bash[17385]: audit 2024-02-10T08:18:58.698803+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:18:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:18:59 smithi067 bash[17385]: audit 2024-02-10T08:18:58.699656+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:18:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:59 smithi134 bash[18722]: audit 2024-02-10T08:18:58.695933+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:18:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:59 smithi134 bash[18722]: audit 2024-02-10T08:18:58.698803+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:18:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:18:59 smithi134 bash[18722]: audit 2024-02-10T08:18:58.699656+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:00 smithi067 bash[17385]: audit 2024-02-10T08:18:58.690146+0000 mgr.y (mgr.14146) 157 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:19:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:00 smithi067 bash[17385]: cluster 2024-02-10T08:18:58.857788+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:00 smithi067 bash[24488]: audit 2024-02-10T08:18:58.690146+0000 mgr.y (mgr.14146) 157 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:19:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:00 smithi067 bash[24488]: cluster 2024-02-10T08:18:58.857788+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:00 smithi134 bash[18722]: audit 2024-02-10T08:18:58.690146+0000 mgr.y (mgr.14146) 157 : audit [DBG] from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:19:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:00 smithi134 bash[18722]: cluster 2024-02-10T08:18:58.857788+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:01.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: cephadm 2024-02-10T08:19:00.586065+0000 mgr.y (mgr.14146) 159 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:19:01.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.596124+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:01.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.597609+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:01.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.598688+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.599817+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: cephadm 2024-02-10T08:19:00.600658+0000 mgr.y (mgr.14146) 160 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 5413M 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.605268+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.606624+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.607935+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: audit 2024-02-10T08:19:00.614322+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[24488]: cluster 2024-02-10T08:19:00.858288+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: cephadm 2024-02-10T08:19:00.586065+0000 mgr.y (mgr.14146) 159 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.596124+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.597609+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.598688+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.599817+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:01.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: cephadm 2024-02-10T08:19:00.600658+0000 mgr.y (mgr.14146) 160 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 5413M 2024-02-10T08:19:01.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.605268+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:01.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.606624+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:01.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.607935+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:01.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: audit 2024-02-10T08:19:00.614322+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:01.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:01 smithi067 bash[17385]: cluster 2024-02-10T08:19:00.858288+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: cephadm 2024-02-10T08:19:00.586065+0000 mgr.y (mgr.14146) 159 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:19:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.596124+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.597609+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.598688+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.599817+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: cephadm 2024-02-10T08:19:00.600658+0000 mgr.y (mgr.14146) 160 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 5413M 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.605268+0000 mon.a (mon.0) 339 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.606624+0000 mon.a (mon.0) 340 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.607935+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: audit 2024-02-10T08:19:00.614322+0000 mon.a (mon.0) 342 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:01 smithi134 bash[18722]: cluster 2024-02-10T08:19:00.858288+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v139: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:04.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:04 smithi067 bash[17385]: cluster 2024-02-10T08:19:02.858844+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:04.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:04 smithi067 bash[24488]: cluster 2024-02-10T08:19:02.858844+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:04 smithi134 bash[18722]: cluster 2024-02-10T08:19:02.858844+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:06 smithi067 bash[17385]: cluster 2024-02-10T08:19:04.859439+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:06 smithi067 bash[24488]: cluster 2024-02-10T08:19:04.859439+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:06 smithi134 bash[18722]: cluster 2024-02-10T08:19:04.859439+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[17385]: cluster 2024-02-10T08:19:06.860060+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[17385]: audit 2024-02-10T08:19:07.475628+0000 mon.a (mon.0) 343 : audit [INF] from='client.? 172.21.15.67:0/2949155332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ddc51e-07b3-4e88-a568-eb7ec0dedbad"}]: dispatch 2024-02-10T08:19:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[17385]: audit 2024-02-10T08:19:07.486385+0000 mon.a (mon.0) 344 : audit [INF] from='client.? 172.21.15.67:0/2949155332' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13ddc51e-07b3-4e88-a568-eb7ec0dedbad"}]': finished 2024-02-10T08:19:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[17385]: cluster 2024-02-10T08:19:07.486456+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-10T08:19:08.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[17385]: audit 2024-02-10T08:19:07.486721+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:08.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[24488]: cluster 2024-02-10T08:19:06.860060+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:08.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[24488]: audit 2024-02-10T08:19:07.475628+0000 mon.a (mon.0) 343 : audit [INF] from='client.? 172.21.15.67:0/2949155332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ddc51e-07b3-4e88-a568-eb7ec0dedbad"}]: dispatch 2024-02-10T08:19:08.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[24488]: audit 2024-02-10T08:19:07.486385+0000 mon.a (mon.0) 344 : audit [INF] from='client.? 172.21.15.67:0/2949155332' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13ddc51e-07b3-4e88-a568-eb7ec0dedbad"}]': finished 2024-02-10T08:19:08.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[24488]: cluster 2024-02-10T08:19:07.486456+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-10T08:19:08.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:08 smithi067 bash[24488]: audit 2024-02-10T08:19:07.486721+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:08.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:08 smithi134 bash[18722]: cluster 2024-02-10T08:19:06.860060+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:08.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:08 smithi134 bash[18722]: audit 2024-02-10T08:19:07.475628+0000 mon.a (mon.0) 343 : audit [INF] from='client.? 172.21.15.67:0/2949155332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13ddc51e-07b3-4e88-a568-eb7ec0dedbad"}]: dispatch 2024-02-10T08:19:08.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:08 smithi134 bash[18722]: audit 2024-02-10T08:19:07.486385+0000 mon.a (mon.0) 344 : audit [INF] from='client.? 172.21.15.67:0/2949155332' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13ddc51e-07b3-4e88-a568-eb7ec0dedbad"}]': finished 2024-02-10T08:19:08.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:08 smithi134 bash[18722]: cluster 2024-02-10T08:19:07.486456+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-10T08:19:08.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:08 smithi134 bash[18722]: audit 2024-02-10T08:19:07.486721+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:09 smithi067 bash[17385]: audit 2024-02-10T08:19:08.228316+0000 mon.a (mon.0) 347 : audit [DBG] from='client.? 172.21.15.67:0/708285863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:19:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:09 smithi067 bash[24488]: audit 2024-02-10T08:19:08.228316+0000 mon.a (mon.0) 347 : audit [DBG] from='client.? 172.21.15.67:0/708285863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:19:09.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:09 smithi134 bash[18722]: audit 2024-02-10T08:19:08.228316+0000 mon.a (mon.0) 347 : audit [DBG] from='client.? 172.21.15.67:0/708285863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:19:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:10 smithi067 bash[17385]: cluster 2024-02-10T08:19:08.860678+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:10 smithi067 bash[24488]: cluster 2024-02-10T08:19:08.860678+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:10.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:10 smithi134 bash[18722]: cluster 2024-02-10T08:19:08.860678+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:12 smithi067 bash[17385]: cluster 2024-02-10T08:19:10.861107+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:12 smithi067 bash[24488]: cluster 2024-02-10T08:19:10.861107+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:12.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:12 smithi134 bash[18722]: cluster 2024-02-10T08:19:10.861107+0000 mgr.y (mgr.14146) 166 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:14 smithi067 bash[17385]: cluster 2024-02-10T08:19:12.861584+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:14 smithi067 bash[24488]: cluster 2024-02-10T08:19:12.861584+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:14 smithi134 bash[18722]: cluster 2024-02-10T08:19:12.861584+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:16.369 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:16 smithi067 bash[24488]: cluster 2024-02-10T08:19:14.862063+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:16.369 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:16 smithi067 bash[17385]: cluster 2024-02-10T08:19:14.862063+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:16.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:16 smithi134 bash[18722]: cluster 2024-02-10T08:19:14.862063+0000 mgr.y (mgr.14146) 168 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:18 smithi067 bash[17385]: cluster 2024-02-10T08:19:16.862679+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:18.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:18 smithi067 bash[24488]: cluster 2024-02-10T08:19:16.862679+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:18 smithi134 bash[18722]: cluster 2024-02-10T08:19:16.862679+0000 mgr.y (mgr.14146) 169 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:20 smithi067 bash[17385]: cluster 2024-02-10T08:19:18.863154+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:20 smithi067 bash[17385]: audit 2024-02-10T08:19:20.008783+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T08:19:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:20 smithi067 bash[17385]: audit 2024-02-10T08:19:20.009906+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:20.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:20 smithi067 bash[24488]: cluster 2024-02-10T08:19:18.863154+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:20.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:20 smithi067 bash[24488]: audit 2024-02-10T08:19:20.008783+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T08:19:20.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:20 smithi067 bash[24488]: audit 2024-02-10T08:19:20.009906+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:20 smithi134 bash[18722]: cluster 2024-02-10T08:19:18.863154+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:20 smithi134 bash[18722]: audit 2024-02-10T08:19:20.008783+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-10T08:19:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:20 smithi134 bash[18722]: audit 2024-02-10T08:19:20.009906+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:21.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:21 smithi067 bash[17385]: cephadm 2024-02-10T08:19:20.010747+0000 mgr.y (mgr.14146) 171 : cephadm [INF] Deploying daemon osd.3 on smithi067 2024-02-10T08:19:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:21 smithi067 bash[24488]: cephadm 2024-02-10T08:19:20.010747+0000 mgr.y (mgr.14146) 171 : cephadm [INF] Deploying daemon osd.3 on smithi067 2024-02-10T08:19:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:21 smithi134 bash[18722]: cephadm 2024-02-10T08:19:20.010747+0000 mgr.y (mgr.14146) 171 : cephadm [INF] Deploying daemon osd.3 on smithi067 2024-02-10T08:19:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:22 smithi067 bash[17385]: cluster 2024-02-10T08:19:20.863524+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:22.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:22 smithi067 bash[24488]: cluster 2024-02-10T08:19:20.863524+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:22 smithi134 bash[18722]: cluster 2024-02-10T08:19:20.863524+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:24 smithi067 bash[24488]: cluster 2024-02-10T08:19:22.863997+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:24.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:24 smithi067 bash[17385]: cluster 2024-02-10T08:19:22.863997+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:24 smithi134 bash[18722]: cluster 2024-02-10T08:19:22.863997+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:25.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:25 smithi067 bash[24488]: audit 2024-02-10T08:19:24.361802+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:25.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:25 smithi067 bash[24488]: audit 2024-02-10T08:19:24.393391+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:19:25.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:25 smithi067 bash[24488]: cluster 2024-02-10T08:19:24.864458+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:25 smithi067 bash[17385]: audit 2024-02-10T08:19:24.361802+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:25 smithi067 bash[17385]: audit 2024-02-10T08:19:24.393391+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:19:25.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:25 smithi067 bash[17385]: cluster 2024-02-10T08:19:24.864458+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:25 smithi134 bash[18722]: audit 2024-02-10T08:19:24.361802+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:25 smithi134 bash[18722]: audit 2024-02-10T08:19:24.393391+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:19:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:25 smithi134 bash[18722]: cluster 2024-02-10T08:19:24.864458+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:28.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:28 smithi067 bash[24488]: cluster 2024-02-10T08:19:26.864836+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:28.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:28 smithi067 bash[17385]: cluster 2024-02-10T08:19:26.864836+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:28 smithi134 bash[18722]: cluster 2024-02-10T08:19:26.864836+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:30.333 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[24488]: cluster 2024-02-10T08:19:28.865335+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[24488]: audit 2024-02-10T08:19:29.689748+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[24488]: audit 2024-02-10T08:19:29.691168+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[24488]: audit 2024-02-10T08:19:29.692555+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[24488]: audit 2024-02-10T08:19:29.700044+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[17385]: cluster 2024-02-10T08:19:28.865335+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[17385]: audit 2024-02-10T08:19:29.689748+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[17385]: audit 2024-02-10T08:19:29.691168+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[17385]: audit 2024-02-10T08:19:29.692555+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:30.334 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:30 smithi067 bash[17385]: audit 2024-02-10T08:19:29.700044+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:30 smithi134 bash[18722]: cluster 2024-02-10T08:19:28.865335+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:30 smithi134 bash[18722]: audit 2024-02-10T08:19:29.689748+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:30 smithi134 bash[18722]: audit 2024-02-10T08:19:29.691168+0000 mon.a (mon.0) 353 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:30 smithi134 bash[18722]: audit 2024-02-10T08:19:29.692555+0000 mon.a (mon.0) 354 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:30.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:30 smithi134 bash[18722]: audit 2024-02-10T08:19:29.700044+0000 mon.a (mon.0) 355 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:32 smithi067 bash[17385]: cluster 2024-02-10T08:19:30.865875+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:32 smithi067 bash[17385]: audit 2024-02-10T08:19:31.142652+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T08:19:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:32 smithi067 bash[24488]: cluster 2024-02-10T08:19:30.865875+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:32 smithi067 bash[24488]: audit 2024-02-10T08:19:31.142652+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T08:19:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:32 smithi134 bash[18722]: cluster 2024-02-10T08:19:30.865875+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:32 smithi134 bash[18722]: audit 2024-02-10T08:19:31.142652+0000 mon.a (mon.0) 356 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-10T08:19:33.260 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.252+0000 7f9487fff700 1 -- 172.21.15.67:0/2325809425 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f94a0101000 con 0x7f9488061250 2024-02-10T08:19:33.260 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 3 on host 'smithi067' 2024-02-10T08:19:33.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9488061250 msgr2=0x7f9488063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:19:33.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9488061250 0x7f9488063710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f94a00fae30 tx=0x7f949400b040).stop 2024-02-10T08:19:33.261 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 msgr2=0x7f94a01114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a01114b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f9498009be0 tx=0x7f9498004d10).stop 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 shutdown_connections 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9488061250 0x7f9488063710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f94a00fb920 0x7f94a01114b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94a00f9d30 0x7f94a0109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 --2- 172.21.15.67:0/2325809425 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f94a00fafd0 0x7f94a010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 >> 172.21.15.67:0/2325809425 conn(0x7f94a00f5710 msgr2=0x7f94a00fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 shutdown_connections 2024-02-10T08:19:33.262 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:19:33.256+0000 7f94a56e4700 1 -- 172.21.15.67:0/2325809425 wait complete. 2024-02-10T08:19:33.314 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[17385]: audit 2024-02-10T08:19:32.050410+0000 mon.a (mon.0) 357 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T08:19:33.314 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[17385]: cluster 2024-02-10T08:19:32.050496+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-10T08:19:33.314 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[17385]: audit 2024-02-10T08:19:32.051163+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:33.314 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[17385]: audit 2024-02-10T08:19:32.051709+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:19:33.315 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[24488]: audit 2024-02-10T08:19:32.050410+0000 mon.a (mon.0) 357 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T08:19:33.315 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[24488]: cluster 2024-02-10T08:19:32.050496+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-10T08:19:33.315 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[24488]: audit 2024-02-10T08:19:32.051163+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:33.315 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:33 smithi067 bash[24488]: audit 2024-02-10T08:19:32.051709+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:19:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:33 smithi134 bash[18722]: audit 2024-02-10T08:19:32.050410+0000 mon.a (mon.0) 357 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-10T08:19:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:33 smithi134 bash[18722]: cluster 2024-02-10T08:19:32.050496+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-10T08:19:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:33 smithi134 bash[18722]: audit 2024-02-10T08:19:32.051163+0000 mon.a (mon.0) 359 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:33 smithi134 bash[18722]: audit 2024-02-10T08:19:32.051709+0000 mon.a (mon.0) 360 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-10T08:19:34.084 DEBUG:teuthology.orchestra.run.smithi067:osd.3> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.3.service 2024-02-10T08:19:34.087 INFO:tasks.cephadm:Deploying osd.4 on smithi134 with /dev/vg_nvme/lv_4... 2024-02-10T08:19:34.087 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-10T08:19:34.255 INFO:journalctl@ceph.osd.3.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:19:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: cluster 2024-02-10T08:19:32.866414+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: audit 2024-02-10T08:19:33.052255+0000 mon.a (mon.0) 361 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:19:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: cluster 2024-02-10T08:19:33.052332+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-10T08:19:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: audit 2024-02-10T08:19:33.052515+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: audit 2024-02-10T08:19:33.060330+0000 mon.a (mon.0) 364 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: audit 2024-02-10T08:19:33.254541+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: audit 2024-02-10T08:19:33.275573+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:19:34.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[24488]: audit 2024-02-10T08:19:34.059106+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: cluster 2024-02-10T08:19:32.866414+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: audit 2024-02-10T08:19:33.052255+0000 mon.a (mon.0) 361 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:19:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: cluster 2024-02-10T08:19:33.052332+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-10T08:19:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: audit 2024-02-10T08:19:33.052515+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: audit 2024-02-10T08:19:33.060330+0000 mon.a (mon.0) 364 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: audit 2024-02-10T08:19:33.254541+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: audit 2024-02-10T08:19:33.275573+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:19:34.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:34 smithi067 bash[17385]: audit 2024-02-10T08:19:34.059106+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: cluster 2024-02-10T08:19:32.866414+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: audit 2024-02-10T08:19:33.052255+0000 mon.a (mon.0) 361 : audit [INF] from='osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-10T08:19:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: cluster 2024-02-10T08:19:33.052332+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-10T08:19:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: audit 2024-02-10T08:19:33.052515+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: audit 2024-02-10T08:19:33.060330+0000 mon.a (mon.0) 364 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: audit 2024-02-10T08:19:33.254541+0000 mon.a (mon.0) 365 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: audit 2024-02-10T08:19:33.275573+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:19:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:34 smithi134 bash[18722]: audit 2024-02-10T08:19:34.059106+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:35.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[17385]: cluster 2024-02-10T08:19:32.110570+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[17385]: cluster 2024-02-10T08:19:32.110699+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[17385]: cluster 2024-02-10T08:19:34.069590+0000 mon.a (mon.0) 368 : cluster [INF] osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] boot 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[17385]: cluster 2024-02-10T08:19:34.069661+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[17385]: audit 2024-02-10T08:19:34.070613+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[24488]: cluster 2024-02-10T08:19:32.110570+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[24488]: cluster 2024-02-10T08:19:32.110699+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[24488]: cluster 2024-02-10T08:19:34.069590+0000 mon.a (mon.0) 368 : cluster [INF] osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] boot 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[24488]: cluster 2024-02-10T08:19:34.069661+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-10T08:19:35.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:35 smithi067 bash[24488]: audit 2024-02-10T08:19:34.070613+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:35.415 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:35 smithi134 bash[18722]: cluster 2024-02-10T08:19:32.110570+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:19:35.415 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:35 smithi134 bash[18722]: cluster 2024-02-10T08:19:32.110699+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:19:35.415 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:35 smithi134 bash[18722]: cluster 2024-02-10T08:19:34.069590+0000 mon.a (mon.0) 368 : cluster [INF] osd.3 [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] boot 2024-02-10T08:19:35.415 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:35 smithi134 bash[18722]: cluster 2024-02-10T08:19:34.069661+0000 mon.a (mon.0) 369 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-10T08:19:35.415 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:35 smithi134 bash[18722]: audit 2024-02-10T08:19:34.070613+0000 mon.a (mon.0) 370 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:19:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:36 smithi134 bash[18722]: cluster 2024-02-10T08:19:34.866964+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:36 smithi134 bash[18722]: cluster 2024-02-10T08:19:35.087132+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-10T08:19:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:36 smithi067 bash[17385]: cluster 2024-02-10T08:19:34.866964+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:36 smithi067 bash[17385]: cluster 2024-02-10T08:19:35.087132+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-10T08:19:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:36 smithi067 bash[24488]: cluster 2024-02-10T08:19:34.866964+0000 mgr.y (mgr.14146) 179 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-10T08:19:36.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:36 smithi067 bash[24488]: cluster 2024-02-10T08:19:35.087132+0000 mon.a (mon.0) 371 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-10T08:19:37.335 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-10T08:19:37.336 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-10T08:19:37.336 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-02-10T08:19:37.336 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-02-10T08:19:37.336 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0159365 s, 658 MB/s 2024-02-10T08:19:37.336 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-02-10T08:19:38.045 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-02-10T08:19:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:38 smithi067 bash[17385]: cluster 2024-02-10T08:19:36.867507+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:38.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:38 smithi067 bash[24488]: cluster 2024-02-10T08:19:36.867507+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:38 smithi134 bash[18722]: cluster 2024-02-10T08:19:36.867507+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:40 smithi134 bash[18722]: cluster 2024-02-10T08:19:38.868036+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:40 smithi067 bash[17385]: cluster 2024-02-10T08:19:38.868036+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:40 smithi067 bash[24488]: cluster 2024-02-10T08:19:38.868036+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- 172.21.15.134:0/145094866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f7b00 msgr2=0x7fdf200f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:19:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- 172.21.15.134:0/145094866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f7b00 0x7fdf200f7f20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fdf1c004870 tx=0x7fdf1c009cb0).stop 2024-02-10T08:19:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- 172.21.15.134:0/145094866 shutdown_connections 2024-02-10T08:19:40.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- 172.21.15.134:0/145094866 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 0x7fdf20100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- 172.21.15.134:0/145094866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f7b00 0x7fdf200f7f20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- 172.21.15.134:0/145094866 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f8410 0x7fdf200f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- 172.21.15.134:0/145094866 >> 172.21.15.134:0/145094866 conn(0x7fdf200f3340 msgr2=0x7fdf200f5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:19:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- 172.21.15.134:0/145094866 shutdown_connections 2024-02-10T08:19:40.682 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- 172.21.15.134:0/145094866 wait complete. 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 Processor -- start 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- start start 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 0x7fdf2006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f8410 0x7fdf2006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 0x7fdf2006b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2006bb90 con 0x7fdf200f7b00 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2006bd00 con 0x7fdf200f8410 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf2872a700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2006bec0 con 0x7fdf200f9520 2024-02-10T08:19:40.683 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf264c6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 0x7fdf2006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf264c6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 0x7fdf2006ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.134:60028/0 (socket says 172.21.15.134:60028) 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf264c6700 1 -- 172.21.15.134:0/370070009 learned_addr learned my addr 172.21.15.134:0/370070009 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf25cc5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f8410 0x7fdf2006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.675+0000 7fdf26cc7700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 0x7fdf2006b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf264c6700 1 -- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 msgr2=0x7fdf2006b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf264c6700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 0x7fdf2006b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf264c6700 1 -- 172.21.15.134:0/370070009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f8410 msgr2=0x7fdf2006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf264c6700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f8410 0x7fdf2006b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf264c6700 1 -- 172.21.15.134:0/370070009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c005040 con 0x7fdf200f7b00 2024-02-10T08:19:40.684 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf26cc7700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 0x7fdf2006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:19:40.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf264c6700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 0x7fdf2006ca20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdf1c013db0 tx=0x7fdf1c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:19:40.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00e030 con 0x7fdf200f7b00 2024-02-10T08:19:40.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf20111d90 con 0x7fdf200f7b00 2024-02-10T08:19:40.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fdf1c01d420 con 0x7fdf200f7b00 2024-02-10T08:19:40.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c018070 con 0x7fdf200f7b00 2024-02-10T08:19:40.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf20112190 con 0x7fdf200f7b00 2024-02-10T08:19:40.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf20060070 con 0x7fdf200f7b00 2024-02-10T08:19:40.686 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.679+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fdf1c009450 con 0x7fdf200f7b00 2024-02-10T08:19:40.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.683+0000 7fdf177fe700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdf0c061250 0x7fdf0c063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:19:40.687 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.683+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3422+0+0 (secure 0 0 0) 0x7fdf1c07d5d0 con 0x7fdf200f7b00 2024-02-10T08:19:40.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.683+0000 7fdf25cc5700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdf0c061250 0x7fdf0c063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:19:40.688 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.683+0000 7fdf25cc5700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdf0c061250 0x7fdf0c063710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fdf10004c20 tx=0x7fdf10009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:19:40.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.691+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fdf1c04e210 con 0x7fdf200f7b00 2024-02-10T08:19:40.890 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:19:40.883+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf200fee80 con 0x7fdf0c061250 2024-02-10T08:19:41.056 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:41 smithi067 bash[24488]: audit 2024-02-10T08:19:40.899317+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:19:41.057 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:41 smithi067 bash[24488]: audit 2024-02-10T08:19:40.903169+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:19:41.057 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:41 smithi067 bash[24488]: audit 2024-02-10T08:19:40.904428+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:41.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:41 smithi067 bash[17385]: audit 2024-02-10T08:19:40.899317+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:19:41.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:41 smithi067 bash[17385]: audit 2024-02-10T08:19:40.903169+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:19:41.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:41 smithi067 bash[17385]: audit 2024-02-10T08:19:40.904428+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:41 smithi134 bash[18722]: audit 2024-02-10T08:19:40.899317+0000 mon.a (mon.0) 372 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:19:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:41 smithi134 bash[18722]: audit 2024-02-10T08:19:40.903169+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:19:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:41 smithi134 bash[18722]: audit 2024-02-10T08:19:40.904428+0000 mon.a (mon.0) 374 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:42 smithi067 bash[17385]: cluster 2024-02-10T08:19:40.868688+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:42 smithi067 bash[17385]: audit 2024-02-10T08:19:40.892056+0000 mgr.y (mgr.14146) 183 : audit [DBG] from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:19:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:42 smithi067 bash[24488]: cluster 2024-02-10T08:19:40.868688+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:42 smithi067 bash[24488]: audit 2024-02-10T08:19:40.892056+0000 mgr.y (mgr.14146) 183 : audit [DBG] from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:19:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:42 smithi134 bash[18722]: cluster 2024-02-10T08:19:40.868688+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:42 smithi134 bash[18722]: audit 2024-02-10T08:19:40.892056+0000 mgr.y (mgr.14146) 183 : audit [DBG] from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:19:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: cephadm 2024-02-10T08:19:42.697693+0000 mgr.y (mgr.14146) 184 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:19:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.707228+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.709006+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.710176+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.711316+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.712614+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: cephadm 2024-02-10T08:19:42.713384+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 4060M 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.718210+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.720092+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.721369+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: audit 2024-02-10T08:19:42.729960+0000 mon.a (mon.0) 383 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:43 smithi134 bash[18722]: cluster 2024-02-10T08:19:42.869225+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: cephadm 2024-02-10T08:19:42.697693+0000 mgr.y (mgr.14146) 184 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:19:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.707228+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.709006+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.710176+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.711316+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.712614+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: cephadm 2024-02-10T08:19:42.713384+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 4060M 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.718210+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.720092+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.721369+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: audit 2024-02-10T08:19:42.729960+0000 mon.a (mon.0) 383 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[17385]: cluster 2024-02-10T08:19:42.869225+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: cephadm 2024-02-10T08:19:42.697693+0000 mgr.y (mgr.14146) 184 : cephadm [INF] Detected new or changed devices on smithi067 2024-02-10T08:19:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.707228+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.709006+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.710176+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.711316+0000 mon.a (mon.0) 378 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.712614+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: cephadm 2024-02-10T08:19:42.713384+0000 mgr.y (mgr.14146) 185 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 4060M 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.718210+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.720092+0000 mon.a (mon.0) 381 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.721369+0000 mon.a (mon.0) 382 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: audit 2024-02-10T08:19:42.729960+0000 mon.a (mon.0) 383 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:19:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:43 smithi067 bash[24488]: cluster 2024-02-10T08:19:42.869225+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:45 smithi134 bash[18722]: cluster 2024-02-10T08:19:44.869733+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:45 smithi067 bash[17385]: cluster 2024-02-10T08:19:44.869733+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:45 smithi067 bash[24488]: cluster 2024-02-10T08:19:44.869733+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:48 smithi067 bash[17385]: cluster 2024-02-10T08:19:46.870302+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:48 smithi067 bash[24488]: cluster 2024-02-10T08:19:46.870302+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:48 smithi134 bash[18722]: cluster 2024-02-10T08:19:46.870302+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: cluster 2024-02-10T08:19:48.870865+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: audit 2024-02-10T08:19:49.097004+0000 mon.b (mon.1) 3 : audit [INF] from='client.? 172.21.15.134:0/1914608589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]: dispatch 2024-02-10T08:19:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: audit 2024-02-10T08:19:49.098018+0000 mon.a (mon.0) 384 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]: dispatch 2024-02-10T08:19:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: audit 2024-02-10T08:19:49.106323+0000 mon.a (mon.0) 385 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]': finished 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: cluster 2024-02-10T08:19:49.106600+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: audit 2024-02-10T08:19:49.106864+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[17385]: audit 2024-02-10T08:19:49.806073+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? 172.21.15.134:0/190639069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: cluster 2024-02-10T08:19:48.870865+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: audit 2024-02-10T08:19:49.097004+0000 mon.b (mon.1) 3 : audit [INF] from='client.? 172.21.15.134:0/1914608589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]: dispatch 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: audit 2024-02-10T08:19:49.098018+0000 mon.a (mon.0) 384 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]: dispatch 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: audit 2024-02-10T08:19:49.106323+0000 mon.a (mon.0) 385 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]': finished 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: cluster 2024-02-10T08:19:49.106600+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: audit 2024-02-10T08:19:49.106864+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:19:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:50 smithi067 bash[24488]: audit 2024-02-10T08:19:49.806073+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? 172.21.15.134:0/190639069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:19:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: cluster 2024-02-10T08:19:48.870865+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: audit 2024-02-10T08:19:49.097004+0000 mon.b (mon.1) 3 : audit [INF] from='client.? 172.21.15.134:0/1914608589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]: dispatch 2024-02-10T08:19:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: audit 2024-02-10T08:19:49.098018+0000 mon.a (mon.0) 384 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]: dispatch 2024-02-10T08:19:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: audit 2024-02-10T08:19:49.106323+0000 mon.a (mon.0) 385 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5a34154-4009-4d77-a6f1-7b1c6a7cdd12"}]': finished 2024-02-10T08:19:50.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: cluster 2024-02-10T08:19:49.106600+0000 mon.a (mon.0) 386 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-10T08:19:50.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: audit 2024-02-10T08:19:49.106864+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:19:50.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:50 smithi134 bash[18722]: audit 2024-02-10T08:19:49.806073+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? 172.21.15.134:0/190639069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:19:52.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:52 smithi134 bash[18722]: cluster 2024-02-10T08:19:50.871430+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:52 smithi067 bash[17385]: cluster 2024-02-10T08:19:50.871430+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:52 smithi067 bash[24488]: cluster 2024-02-10T08:19:50.871430+0000 mgr.y (mgr.14146) 190 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:54.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:54 smithi067 bash[17385]: cluster 2024-02-10T08:19:52.871967+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:54 smithi067 bash[24488]: cluster 2024-02-10T08:19:52.871967+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:54 smithi134 bash[18722]: cluster 2024-02-10T08:19:52.871967+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:56 smithi067 bash[17385]: cluster 2024-02-10T08:19:54.872492+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:56 smithi067 bash[24488]: cluster 2024-02-10T08:19:54.872492+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:56 smithi134 bash[18722]: cluster 2024-02-10T08:19:54.872492+0000 mgr.y (mgr.14146) 192 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:19:58 smithi134 bash[18722]: cluster 2024-02-10T08:19:56.873056+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:58.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:19:58 smithi067 bash[24488]: cluster 2024-02-10T08:19:56.873056+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:19:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:19:58 smithi067 bash[17385]: cluster 2024-02-10T08:19:56.873056+0000 mgr.y (mgr.14146) 193 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:00 smithi067 bash[17385]: cluster 2024-02-10T08:19:58.873703+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:00 smithi067 bash[17385]: cluster 2024-02-10T08:20:00.000157+0000 mon.a (mon.0) 388 : cluster [INF] overall HEALTH_OK 2024-02-10T08:20:00.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:00 smithi067 bash[24488]: cluster 2024-02-10T08:19:58.873703+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:00.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:00 smithi067 bash[24488]: cluster 2024-02-10T08:20:00.000157+0000 mon.a (mon.0) 388 : cluster [INF] overall HEALTH_OK 2024-02-10T08:20:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:00 smithi134 bash[18722]: cluster 2024-02-10T08:19:58.873703+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:00 smithi134 bash[18722]: cluster 2024-02-10T08:20:00.000157+0000 mon.a (mon.0) 388 : cluster [INF] overall HEALTH_OK 2024-02-10T08:20:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:02 smithi067 bash[17385]: cluster 2024-02-10T08:20:00.874293+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:02 smithi067 bash[17385]: audit 2024-02-10T08:20:01.486955+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T08:20:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:02 smithi067 bash[17385]: audit 2024-02-10T08:20:01.488811+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:02 smithi067 bash[24488]: cluster 2024-02-10T08:20:00.874293+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:02 smithi067 bash[24488]: audit 2024-02-10T08:20:01.486955+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T08:20:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:02 smithi067 bash[24488]: audit 2024-02-10T08:20:01.488811+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:02 smithi134 bash[18722]: cluster 2024-02-10T08:20:00.874293+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:02 smithi134 bash[18722]: audit 2024-02-10T08:20:01.486955+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-10T08:20:02.437 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:02 smithi134 bash[18722]: audit 2024-02-10T08:20:01.488811+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:03.359 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:03 smithi134 bash[18722]: cephadm 2024-02-10T08:20:01.490359+0000 mgr.y (mgr.14146) 196 : cephadm [INF] Deploying daemon osd.4 on smithi134 2024-02-10T08:20:03.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:03 smithi067 bash[17385]: cephadm 2024-02-10T08:20:01.490359+0000 mgr.y (mgr.14146) 196 : cephadm [INF] Deploying daemon osd.4 on smithi134 2024-02-10T08:20:03.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:03 smithi067 bash[24488]: cephadm 2024-02-10T08:20:01.490359+0000 mgr.y (mgr.14146) 196 : cephadm [INF] Deploying daemon osd.4 on smithi134 2024-02-10T08:20:04.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:04 smithi067 bash[17385]: cluster 2024-02-10T08:20:02.874813+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:04.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:04 smithi067 bash[24488]: cluster 2024-02-10T08:20:02.874813+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:04.463 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:04 smithi134 bash[18722]: cluster 2024-02-10T08:20:02.874813+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:05 smithi067 bash[17385]: cluster 2024-02-10T08:20:04.875312+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:05 smithi067 bash[24488]: cluster 2024-02-10T08:20:04.875312+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:05 smithi134 bash[18722]: cluster 2024-02-10T08:20:04.875312+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:06 smithi134 bash[18722]: audit 2024-02-10T08:20:05.591622+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:06 smithi134 bash[18722]: audit 2024-02-10T08:20:05.609193+0000 mon.a (mon.0) 392 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:06 smithi067 bash[17385]: audit 2024-02-10T08:20:05.591622+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:06 smithi067 bash[17385]: audit 2024-02-10T08:20:05.609193+0000 mon.a (mon.0) 392 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:06 smithi067 bash[24488]: audit 2024-02-10T08:20:05.591622+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:06 smithi067 bash[24488]: audit 2024-02-10T08:20:05.609193+0000 mon.a (mon.0) 392 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:08.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:08 smithi067 bash[17385]: cluster 2024-02-10T08:20:06.875893+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:08.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:08 smithi067 bash[24488]: cluster 2024-02-10T08:20:06.875893+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:08.397 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:08 smithi134 bash[18722]: cluster 2024-02-10T08:20:06.875893+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:10.358 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:10 smithi134 bash[18722]: cluster 2024-02-10T08:20:08.876464+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:10 smithi067 bash[24488]: cluster 2024-02-10T08:20:08.876464+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:10 smithi067 bash[17385]: cluster 2024-02-10T08:20:08.876464+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v179: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:11 smithi134 bash[18722]: audit 2024-02-10T08:20:10.652869+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:11 smithi134 bash[18722]: cluster 2024-02-10T08:20:10.876992+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:11 smithi134 bash[18722]: audit 2024-02-10T08:20:10.929544+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:11.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:11 smithi134 bash[18722]: audit 2024-02-10T08:20:10.931376+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:11.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:11 smithi134 bash[18722]: audit 2024-02-10T08:20:10.941254+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[17385]: audit 2024-02-10T08:20:10.652869+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[17385]: cluster 2024-02-10T08:20:10.876992+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[17385]: audit 2024-02-10T08:20:10.929544+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[17385]: audit 2024-02-10T08:20:10.931376+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[17385]: audit 2024-02-10T08:20:10.941254+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[24488]: audit 2024-02-10T08:20:10.652869+0000 mon.a (mon.0) 393 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[24488]: cluster 2024-02-10T08:20:10.876992+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[24488]: audit 2024-02-10T08:20:10.929544+0000 mon.a (mon.0) 394 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[24488]: audit 2024-02-10T08:20:10.931376+0000 mon.a (mon.0) 395 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:11 smithi067 bash[24488]: audit 2024-02-10T08:20:10.941254+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:13.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:12 smithi134 bash[18722]: audit 2024-02-10T08:20:12.008711+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T08:20:13.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:12 smithi134 bash[18722]: audit 2024-02-10T08:20:12.009734+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T08:20:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:12 smithi067 bash[24488]: audit 2024-02-10T08:20:12.008711+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T08:20:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:12 smithi067 bash[24488]: audit 2024-02-10T08:20:12.009734+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T08:20:13.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:12 smithi067 bash[17385]: audit 2024-02-10T08:20:12.008711+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T08:20:13.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:12 smithi067 bash[17385]: audit 2024-02-10T08:20:12.009734+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-10T08:20:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:13 smithi134 bash[18722]: audit 2024-02-10T08:20:12.672489+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T08:20:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:13 smithi134 bash[18722]: cluster 2024-02-10T08:20:12.672546+0000 mon.a (mon.0) 399 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-10T08:20:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:13 smithi134 bash[18722]: audit 2024-02-10T08:20:12.672729+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:13 smithi134 bash[18722]: audit 2024-02-10T08:20:12.673336+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:13 smithi134 bash[18722]: audit 2024-02-10T08:20:12.674300+0000 mon.a (mon.0) 401 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:13.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:13 smithi134 bash[18722]: cluster 2024-02-10T08:20:12.877556+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:13.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.978+0000 7fdf177fe700 1 -- 172.21.15.134:0/370070009 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdf200fee80 con 0x7fdf0c061250 2024-02-10T08:20:13.981 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 4 on host 'smithi134' 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdf0c061250 msgr2=0x7fdf0c063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdf0c061250 0x7fdf0c063710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fdf10004c20 tx=0x7fdf10009040).stop 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 msgr2=0x7fdf2006ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 0x7fdf2006ca20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdf1c013db0 tx=0x7fdf1c009cb0).stop 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 shutdown_connections 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fdf0c061250 0x7fdf0c063710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fdf200f7b00 0x7fdf2006ca20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fdf200f9520 0x7fdf2006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:13.985 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 --2- 172.21.15.134:0/370070009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf200f8410 0x7fdf2006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:13.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 >> 172.21.15.134:0/370070009 conn(0x7fdf200f3340 msgr2=0x7fdf200fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:20:13.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 shutdown_connections 2024-02-10T08:20:13.986 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:13.982+0000 7fdf2872a700 1 -- 172.21.15.134:0/370070009 wait complete. 2024-02-10T08:20:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[17385]: audit 2024-02-10T08:20:12.672489+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T08:20:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[17385]: cluster 2024-02-10T08:20:12.672546+0000 mon.a (mon.0) 399 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-10T08:20:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[17385]: audit 2024-02-10T08:20:12.672729+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[17385]: audit 2024-02-10T08:20:12.673336+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[17385]: audit 2024-02-10T08:20:12.674300+0000 mon.a (mon.0) 401 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[17385]: cluster 2024-02-10T08:20:12.877556+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[24488]: audit 2024-02-10T08:20:12.672489+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[24488]: cluster 2024-02-10T08:20:12.672546+0000 mon.a (mon.0) 399 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[24488]: audit 2024-02-10T08:20:12.672729+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[24488]: audit 2024-02-10T08:20:12.673336+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[24488]: audit 2024-02-10T08:20:12.674300+0000 mon.a (mon.0) 401 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:13 smithi067 bash[24488]: cluster 2024-02-10T08:20:12.877556+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-10T08:20:14.673 DEBUG:teuthology.orchestra.run.smithi134:osd.4> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.4.service 2024-02-10T08:20:14.676 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-02-10T08:20:14.676 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-10T08:20:14.872 INFO:journalctl@ceph.osd.4.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:20:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:14 smithi134 bash[18722]: audit 2024-02-10T08:20:13.678793+0000 mon.a (mon.0) 402 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:20:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:14 smithi134 bash[18722]: cluster 2024-02-10T08:20:13.678903+0000 mon.a (mon.0) 403 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-10T08:20:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:14 smithi134 bash[18722]: audit 2024-02-10T08:20:13.679658+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:14 smithi134 bash[18722]: audit 2024-02-10T08:20:13.686746+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:14 smithi134 bash[18722]: audit 2024-02-10T08:20:13.979419+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:15.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:14 smithi134 bash[18722]: audit 2024-02-10T08:20:14.017095+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[17385]: audit 2024-02-10T08:20:13.678793+0000 mon.a (mon.0) 402 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:20:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[17385]: cluster 2024-02-10T08:20:13.678903+0000 mon.a (mon.0) 403 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-10T08:20:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[17385]: audit 2024-02-10T08:20:13.679658+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[17385]: audit 2024-02-10T08:20:13.686746+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[17385]: audit 2024-02-10T08:20:13.979419+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[17385]: audit 2024-02-10T08:20:14.017095+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[24488]: audit 2024-02-10T08:20:13.678793+0000 mon.a (mon.0) 402 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[24488]: cluster 2024-02-10T08:20:13.678903+0000 mon.a (mon.0) 403 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[24488]: audit 2024-02-10T08:20:13.679658+0000 mon.a (mon.0) 404 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[24488]: audit 2024-02-10T08:20:13.686746+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[24488]: audit 2024-02-10T08:20:13.979419+0000 mon.a (mon.0) 406 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:14 smithi067 bash[24488]: audit 2024-02-10T08:20:14.017095+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: cluster 2024-02-10T08:20:12.986297+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: cluster 2024-02-10T08:20:12.986414+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: cluster 2024-02-10T08:20:14.681627+0000 mon.a (mon.0) 408 : cluster [INF] osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] boot 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: cluster 2024-02-10T08:20:14.681709+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: audit 2024-02-10T08:20:14.681977+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: cluster 2024-02-10T08:20:14.878074+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:15.975 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:15 smithi134 bash[18722]: cluster 2024-02-10T08:20:15.683704+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-10T08:20:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: cluster 2024-02-10T08:20:12.986297+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:20:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: cluster 2024-02-10T08:20:12.986414+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:20:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: cluster 2024-02-10T08:20:14.681627+0000 mon.a (mon.0) 408 : cluster [INF] osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] boot 2024-02-10T08:20:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: cluster 2024-02-10T08:20:14.681709+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: audit 2024-02-10T08:20:14.681977+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: cluster 2024-02-10T08:20:14.878074+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[24488]: cluster 2024-02-10T08:20:15.683704+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: cluster 2024-02-10T08:20:12.986297+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: cluster 2024-02-10T08:20:12.986414+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: cluster 2024-02-10T08:20:14.681627+0000 mon.a (mon.0) 408 : cluster [INF] osd.4 [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] boot 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: cluster 2024-02-10T08:20:14.681709+0000 mon.a (mon.0) 409 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: audit 2024-02-10T08:20:14.681977+0000 mon.a (mon.0) 410 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: cluster 2024-02-10T08:20:14.878074+0000 mgr.y (mgr.14146) 203 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:15 smithi067 bash[17385]: cluster 2024-02-10T08:20:15.683704+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-10T08:20:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-10T08:20:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-10T08:20:17.792 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-02-10T08:20:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-02-10T08:20:17.793 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150581 s, 696 MB/s 2024-02-10T08:20:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-02-10T08:20:18.016 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:17 smithi134 bash[18722]: cluster 2024-02-10T08:20:16.685862+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-10T08:20:18.017 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:17 smithi134 bash[18722]: cluster 2024-02-10T08:20:16.878596+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v188: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:18.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:17 smithi067 bash[24488]: cluster 2024-02-10T08:20:16.685862+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-10T08:20:18.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:17 smithi067 bash[24488]: cluster 2024-02-10T08:20:16.878596+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v188: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:17 smithi067 bash[17385]: cluster 2024-02-10T08:20:16.685862+0000 mon.a (mon.0) 412 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-10T08:20:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:17 smithi067 bash[17385]: cluster 2024-02-10T08:20:16.878596+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v188: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:18.511 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-02-10T08:20:20.176 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:19 smithi134 bash[18722]: cluster 2024-02-10T08:20:18.879148+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v189: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:20.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:19 smithi067 bash[24488]: cluster 2024-02-10T08:20:18.879148+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v189: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:19 smithi067 bash[17385]: cluster 2024-02-10T08:20:18.879148+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v189: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:22.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:21 smithi134 bash[18722]: cluster 2024-02-10T08:20:20.879627+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v190: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:21 smithi067 bash[17385]: cluster 2024-02-10T08:20:20.879627+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v190: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:22.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:21 smithi067 bash[24488]: cluster 2024-02-10T08:20:20.879627+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v190: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:22.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 -- 172.21.15.134:0/772447349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 msgr2=0x7f974c0ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:22.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 --2- 172.21.15.134:0/772447349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c0ff470 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f9734004870 tx=0x7f9734009cb0).stop 2024-02-10T08:20:22.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 -- 172.21.15.134:0/772447349 shutdown_connections 2024-02-10T08:20:22.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 --2- 172.21.15.134:0/772447349 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f974c0ff960 0x7f974c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:22.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 --2- 172.21.15.134:0/772447349 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0616f0 0x7f974c0fcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:22.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 --2- 172.21.15.134:0/772447349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c0ff470 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:22.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 -- 172.21.15.134:0/772447349 >> 172.21.15.134:0/772447349 conn(0x7f974c0f2230 msgr2=0x7f974c0f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:20:22.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 -- 172.21.15.134:0/772447349 shutdown_connections 2024-02-10T08:20:22.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.654+0000 7f9750f1b700 1 -- 172.21.15.134:0/772447349 wait complete. 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 Processor -- start 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- start start 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f974c0616f0 0x7f974c10a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c10a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0ff960 0x7f974c1116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c111b90 con 0x7f974c0616f0 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c111cd0 con 0x7f974c0fd080 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c111e10 con 0x7f974c0ff960 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c10a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c10a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:49624/0 (socket says 172.21.15.134:49624) 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 -- 172.21.15.134:0/3191616720 learned_addr learned my addr 172.21.15.134:0/3191616720 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:20:22.661 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f974a59c700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f974c0616f0 0x7f974c10a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f974ad9d700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0ff960 0x7f974c1116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 -- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0ff960 msgr2=0x7f974c1116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0ff960 0x7f974c1116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 -- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f974c0616f0 msgr2=0x7f974c10a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f974c0616f0 0x7f974c10a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 -- 172.21.15.134:0/3191616720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734005040 con 0x7f974c0fd080 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f974ad9d700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0ff960 0x7f974c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9743fff700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c10a5c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9734004710 tx=0x7f9734009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400e030 con 0x7f974c0fd080 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f97340096d0 con 0x7f974c0fd080 2024-02-10T08:20:22.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734018070 con 0x7f974c0fd080 2024-02-10T08:20:22.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c112090 con 0x7f974c0fd080 2024-02-10T08:20:22.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c1124c0 con 0x7f974c0fd080 2024-02-10T08:20:22.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.658+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c05f400 con 0x7f974c0fd080 2024-02-10T08:20:22.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.662+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f9734009240 con 0x7f974c0fd080 2024-02-10T08:20:22.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.662+0000 7f97437fe700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9738061250 0x7f9738063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:20:22.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.662+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3964+0+0 (secure 0 0 0) 0x7f973407d180 con 0x7f974c0fd080 2024-02-10T08:20:22.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.662+0000 7f974a59c700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9738061250 0x7f9738063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:20:22.665 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.662+0000 7f974a59c700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9738061250 0x7f9738063710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f973c009250 tx=0x7f973c009de0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:20:22.669 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.666+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f973404dd00 con 0x7f974c0fd080 2024-02-10T08:20:22.928 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:22.926+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f974c0591c0 con 0x7f9738061250 2024-02-10T08:20:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:23 smithi067 bash[17385]: audit 2024-02-10T08:20:22.936333+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:20:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:23 smithi067 bash[17385]: audit 2024-02-10T08:20:22.939995+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:20:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:23 smithi067 bash[17385]: audit 2024-02-10T08:20:22.941206+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:23 smithi067 bash[24488]: audit 2024-02-10T08:20:22.936333+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:20:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:23 smithi067 bash[24488]: audit 2024-02-10T08:20:22.939995+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:20:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:23 smithi067 bash[24488]: audit 2024-02-10T08:20:22.941206+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:23.490 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:23 smithi134 bash[18722]: audit 2024-02-10T08:20:22.936333+0000 mon.a (mon.0) 413 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:20:23.490 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:23 smithi134 bash[18722]: audit 2024-02-10T08:20:22.939995+0000 mon.a (mon.0) 414 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:20:23.490 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:23 smithi134 bash[18722]: audit 2024-02-10T08:20:22.941206+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:24 smithi067 bash[17385]: cluster 2024-02-10T08:20:22.880195+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:24 smithi067 bash[17385]: audit 2024-02-10T08:20:22.929721+0000 mgr.y (mgr.14146) 208 : audit [DBG] from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:20:24.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:24 smithi067 bash[24488]: cluster 2024-02-10T08:20:22.880195+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:24 smithi067 bash[24488]: audit 2024-02-10T08:20:22.929721+0000 mgr.y (mgr.14146) 208 : audit [DBG] from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:20:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:24 smithi134 bash[18722]: cluster 2024-02-10T08:20:22.880195+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:24 smithi134 bash[18722]: audit 2024-02-10T08:20:22.929721+0000 mgr.y (mgr.14146) 208 : audit [DBG] from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:20:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: cephadm 2024-02-10T08:20:24.405997+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: audit 2024-02-10T08:20:24.415670+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: audit 2024-02-10T08:20:24.417218+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: cephadm 2024-02-10T08:20:24.418466+0000 mgr.y (mgr.14146) 210 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 17265M 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: audit 2024-02-10T08:20:24.427537+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: audit 2024-02-10T08:20:24.429241+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: audit 2024-02-10T08:20:24.430586+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: audit 2024-02-10T08:20:24.440628+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:25 smithi134 bash[18722]: cluster 2024-02-10T08:20:24.880724+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: cephadm 2024-02-10T08:20:24.405997+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:20:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: audit 2024-02-10T08:20:24.415670+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: audit 2024-02-10T08:20:24.417218+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:20:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: cephadm 2024-02-10T08:20:24.418466+0000 mgr.y (mgr.14146) 210 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 17265M 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: audit 2024-02-10T08:20:24.427537+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: audit 2024-02-10T08:20:24.429241+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: audit 2024-02-10T08:20:24.430586+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: audit 2024-02-10T08:20:24.440628+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[17385]: cluster 2024-02-10T08:20:24.880724+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: cephadm 2024-02-10T08:20:24.405997+0000 mgr.y (mgr.14146) 209 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: audit 2024-02-10T08:20:24.415670+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: audit 2024-02-10T08:20:24.417218+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: cephadm 2024-02-10T08:20:24.418466+0000 mgr.y (mgr.14146) 210 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 17265M 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: audit 2024-02-10T08:20:24.427537+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: audit 2024-02-10T08:20:24.429241+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:25.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: audit 2024-02-10T08:20:24.430586+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:25.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: audit 2024-02-10T08:20:24.440628+0000 mon.a (mon.0) 421 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:25.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:25 smithi067 bash[24488]: cluster 2024-02-10T08:20:24.880724+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:28.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:28 smithi067 bash[17385]: cluster 2024-02-10T08:20:26.881273+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:28.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:28 smithi067 bash[24488]: cluster 2024-02-10T08:20:26.881273+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:28.518 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:28 smithi134 bash[18722]: cluster 2024-02-10T08:20:26.881273+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:30 smithi067 bash[17385]: cluster 2024-02-10T08:20:28.881825+0000 mgr.y (mgr.14146) 213 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:30 smithi067 bash[24488]: cluster 2024-02-10T08:20:28.881825+0000 mgr.y (mgr.14146) 213 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:30 smithi134 bash[18722]: cluster 2024-02-10T08:20:28.881825+0000 mgr.y (mgr.14146) 213 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[17385]: audit 2024-02-10T08:20:30.951658+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.134:0/1052414303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]: dispatch 2024-02-10T08:20:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[17385]: audit 2024-02-10T08:20:30.952785+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]: dispatch 2024-02-10T08:20:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[17385]: audit 2024-02-10T08:20:30.961414+0000 mon.a (mon.0) 423 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]': finished 2024-02-10T08:20:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[17385]: cluster 2024-02-10T08:20:30.961649+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-02-10T08:20:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[17385]: audit 2024-02-10T08:20:30.961970+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[24488]: audit 2024-02-10T08:20:30.951658+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.134:0/1052414303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]: dispatch 2024-02-10T08:20:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[24488]: audit 2024-02-10T08:20:30.952785+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]: dispatch 2024-02-10T08:20:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[24488]: audit 2024-02-10T08:20:30.961414+0000 mon.a (mon.0) 423 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]': finished 2024-02-10T08:20:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[24488]: cluster 2024-02-10T08:20:30.961649+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-02-10T08:20:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:31 smithi067 bash[24488]: audit 2024-02-10T08:20:30.961970+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:31 smithi134 bash[18722]: audit 2024-02-10T08:20:30.951658+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.134:0/1052414303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]: dispatch 2024-02-10T08:20:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:31 smithi134 bash[18722]: audit 2024-02-10T08:20:30.952785+0000 mon.a (mon.0) 422 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]: dispatch 2024-02-10T08:20:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:31 smithi134 bash[18722]: audit 2024-02-10T08:20:30.961414+0000 mon.a (mon.0) 423 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f79bc97-f76e-45d8-95e9-06654d75441a"}]': finished 2024-02-10T08:20:31.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:31 smithi134 bash[18722]: cluster 2024-02-10T08:20:30.961649+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-02-10T08:20:31.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:31 smithi134 bash[18722]: audit 2024-02-10T08:20:30.961970+0000 mon.a (mon.0) 425 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:32 smithi067 bash[17385]: cluster 2024-02-10T08:20:30.882406+0000 mgr.y (mgr.14146) 214 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:32 smithi067 bash[17385]: audit 2024-02-10T08:20:31.690251+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.134:0/1708987525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:20:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:32 smithi067 bash[24488]: cluster 2024-02-10T08:20:30.882406+0000 mgr.y (mgr.14146) 214 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:32 smithi067 bash[24488]: audit 2024-02-10T08:20:31.690251+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.134:0/1708987525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:20:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:32 smithi134 bash[18722]: cluster 2024-02-10T08:20:30.882406+0000 mgr.y (mgr.14146) 214 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:32 smithi134 bash[18722]: audit 2024-02-10T08:20:31.690251+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.134:0/1708987525' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:20:34.330 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:34 smithi134 bash[18722]: cluster 2024-02-10T08:20:32.883136+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:34 smithi067 bash[17385]: cluster 2024-02-10T08:20:32.883136+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:34 smithi067 bash[24488]: cluster 2024-02-10T08:20:32.883136+0000 mgr.y (mgr.14146) 215 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:36 smithi067 bash[17385]: cluster 2024-02-10T08:20:34.883735+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:36.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:36 smithi067 bash[24488]: cluster 2024-02-10T08:20:34.883735+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:36.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:36 smithi134 bash[18722]: cluster 2024-02-10T08:20:34.883735+0000 mgr.y (mgr.14146) 216 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:38.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:38 smithi067 bash[17385]: cluster 2024-02-10T08:20:36.884277+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:38.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:38 smithi067 bash[24488]: cluster 2024-02-10T08:20:36.884277+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:38 smithi134 bash[18722]: cluster 2024-02-10T08:20:36.884277+0000 mgr.y (mgr.14146) 217 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:39 smithi134 bash[18722]: cluster 2024-02-10T08:20:38.884772+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:39 smithi067 bash[17385]: cluster 2024-02-10T08:20:38.884772+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:39 smithi067 bash[24488]: cluster 2024-02-10T08:20:38.884772+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:41 smithi134 bash[18722]: cluster 2024-02-10T08:20:40.885331+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:41 smithi067 bash[17385]: cluster 2024-02-10T08:20:40.885331+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:41 smithi067 bash[24488]: cluster 2024-02-10T08:20:40.885331+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:44 smithi067 bash[17385]: cluster 2024-02-10T08:20:42.885880+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:44 smithi067 bash[17385]: audit 2024-02-10T08:20:43.131857+0000 mon.a (mon.0) 426 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T08:20:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:44 smithi067 bash[17385]: audit 2024-02-10T08:20:43.133553+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:44 smithi067 bash[24488]: cluster 2024-02-10T08:20:42.885880+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:44 smithi067 bash[24488]: audit 2024-02-10T08:20:43.131857+0000 mon.a (mon.0) 426 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T08:20:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:44 smithi067 bash[24488]: audit 2024-02-10T08:20:43.133553+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:44.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:44 smithi134 bash[18722]: cluster 2024-02-10T08:20:42.885880+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:44.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:44 smithi134 bash[18722]: audit 2024-02-10T08:20:43.131857+0000 mon.a (mon.0) 426 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-10T08:20:44.438 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:44 smithi134 bash[18722]: audit 2024-02-10T08:20:43.133553+0000 mon.a (mon.0) 427 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:45 smithi134 bash[18722]: cephadm 2024-02-10T08:20:43.135087+0000 mgr.y (mgr.14146) 221 : cephadm [INF] Deploying daemon osd.5 on smithi134 2024-02-10T08:20:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:45 smithi067 bash[24488]: cephadm 2024-02-10T08:20:43.135087+0000 mgr.y (mgr.14146) 221 : cephadm [INF] Deploying daemon osd.5 on smithi134 2024-02-10T08:20:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:45 smithi067 bash[17385]: cephadm 2024-02-10T08:20:43.135087+0000 mgr.y (mgr.14146) 221 : cephadm [INF] Deploying daemon osd.5 on smithi134 2024-02-10T08:20:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:46 smithi067 bash[24488]: cluster 2024-02-10T08:20:44.886441+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:46 smithi067 bash[17385]: cluster 2024-02-10T08:20:44.886441+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:46.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:46 smithi134 bash[18722]: cluster 2024-02-10T08:20:44.886441+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:48 smithi134 bash[18722]: cluster 2024-02-10T08:20:46.887016+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:48 smithi134 bash[18722]: audit 2024-02-10T08:20:47.241647+0000 mon.a (mon.0) 428 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:48 smithi134 bash[18722]: audit 2024-02-10T08:20:47.271827+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:48 smithi067 bash[17385]: cluster 2024-02-10T08:20:46.887016+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:48 smithi067 bash[17385]: audit 2024-02-10T08:20:47.241647+0000 mon.a (mon.0) 428 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:48 smithi067 bash[17385]: audit 2024-02-10T08:20:47.271827+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:48 smithi067 bash[24488]: cluster 2024-02-10T08:20:46.887016+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:48 smithi067 bash[24488]: audit 2024-02-10T08:20:47.241647+0000 mon.a (mon.0) 428 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:48 smithi067 bash[24488]: audit 2024-02-10T08:20:47.271827+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:50.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:50 smithi134 bash[18722]: cluster 2024-02-10T08:20:48.887573+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:50 smithi067 bash[17385]: cluster 2024-02-10T08:20:48.887573+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:50.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:50 smithi067 bash[24488]: cluster 2024-02-10T08:20:48.887573+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:52 smithi134 bash[18722]: cluster 2024-02-10T08:20:50.888132+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:52 smithi067 bash[17385]: cluster 2024-02-10T08:20:50.888132+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:52 smithi067 bash[24488]: cluster 2024-02-10T08:20:50.888132+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:53 smithi134 bash[18722]: audit 2024-02-10T08:20:52.341913+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:53 smithi134 bash[18722]: audit 2024-02-10T08:20:52.343303+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:53 smithi134 bash[18722]: audit 2024-02-10T08:20:52.344765+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:53.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:53 smithi134 bash[18722]: audit 2024-02-10T08:20:52.353353+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:53.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:53 smithi134 bash[18722]: cluster 2024-02-10T08:20:52.888634+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[24488]: audit 2024-02-10T08:20:52.341913+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[24488]: audit 2024-02-10T08:20:52.343303+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[24488]: audit 2024-02-10T08:20:52.344765+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[24488]: audit 2024-02-10T08:20:52.353353+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[24488]: cluster 2024-02-10T08:20:52.888634+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[17385]: audit 2024-02-10T08:20:52.341913+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[17385]: audit 2024-02-10T08:20:52.343303+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:20:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[17385]: audit 2024-02-10T08:20:52.344765+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:20:53.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[17385]: audit 2024-02-10T08:20:52.353353+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:53.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:53 smithi067 bash[17385]: cluster 2024-02-10T08:20:52.888634+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:54.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:54 smithi067 bash[17385]: audit 2024-02-10T08:20:53.633261+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T08:20:54.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:54 smithi067 bash[17385]: audit 2024-02-10T08:20:53.634321+0000 mon.a (mon.0) 434 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T08:20:54.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:54 smithi067 bash[24488]: audit 2024-02-10T08:20:53.633261+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T08:20:54.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:54 smithi067 bash[24488]: audit 2024-02-10T08:20:53.634321+0000 mon.a (mon.0) 434 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T08:20:54.642 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:54 smithi134 bash[18722]: audit 2024-02-10T08:20:53.633261+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T08:20:54.642 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:54 smithi134 bash[18722]: audit 2024-02-10T08:20:53.634321+0000 mon.a (mon.0) 434 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-10T08:20:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:55 smithi134 bash[18722]: audit 2024-02-10T08:20:54.368136+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T08:20:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:55 smithi134 bash[18722]: cluster 2024-02-10T08:20:54.368204+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-10T08:20:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:55 smithi134 bash[18722]: audit 2024-02-10T08:20:54.368359+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:55 smithi134 bash[18722]: audit 2024-02-10T08:20:54.369617+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:55.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:55 smithi134 bash[18722]: audit 2024-02-10T08:20:54.370522+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:55.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:55 smithi134 bash[18722]: cluster 2024-02-10T08:20:54.889161+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:55.721 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.717+0000 7f97437fe700 1 -- 172.21.15.134:0/3191616720 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f974c0591c0 con 0x7f9738061250 2024-02-10T08:20:55.722 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-02-10T08:20:55.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9738061250 msgr2=0x7f9738063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:55.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9738061250 0x7f9738063710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f973c009250 tx=0x7f973c009de0).stop 2024-02-10T08:20:55.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 msgr2=0x7f974c10a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:20:55.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c10a5c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9734004710 tx=0x7f9734009cb0).stop 2024-02-10T08:20:55.725 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 shutdown_connections 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9738061250 0x7f9738063710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f974c0616f0 0x7f974c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f974c0ff960 0x7f974c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 --2- 172.21.15.134:0/3191616720 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f974c0fd080 0x7f974c10a5c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 >> 172.21.15.134:0/3191616720 conn(0x7f974c0f2230 msgr2=0x7f974c0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 shutdown_connections 2024-02-10T08:20:55.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:20:55.721+0000 7f9750f1b700 1 -- 172.21.15.134:0/3191616720 wait complete. 2024-02-10T08:20:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[17385]: audit 2024-02-10T08:20:54.368136+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T08:20:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[17385]: cluster 2024-02-10T08:20:54.368204+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-10T08:20:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[17385]: audit 2024-02-10T08:20:54.368359+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[17385]: audit 2024-02-10T08:20:54.369617+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[17385]: audit 2024-02-10T08:20:54.370522+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[17385]: cluster 2024-02-10T08:20:54.889161+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[24488]: audit 2024-02-10T08:20:54.368136+0000 mon.a (mon.0) 435 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-10T08:20:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[24488]: cluster 2024-02-10T08:20:54.368204+0000 mon.a (mon.0) 436 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-10T08:20:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[24488]: audit 2024-02-10T08:20:54.368359+0000 mon.a (mon.0) 437 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[24488]: audit 2024-02-10T08:20:54.369617+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[24488]: audit 2024-02-10T08:20:54.370522+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:20:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:55 smithi067 bash[24488]: cluster 2024-02-10T08:20:54.889161+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:56.419 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.5.service 2024-02-10T08:20:56.421 INFO:tasks.cephadm:Deploying osd.6 on smithi134 with /dev/vg_nvme/lv_2... 2024-02-10T08:20:56.422 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-10T08:20:56.633 INFO:journalctl@ceph.osd.5.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:20:56.664 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:56 smithi134 bash[18722]: audit 2024-02-10T08:20:55.373095+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:20:56.664 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:56 smithi134 bash[18722]: cluster 2024-02-10T08:20:55.373234+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-10T08:20:56.664 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:56 smithi134 bash[18722]: audit 2024-02-10T08:20:55.374642+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:56.664 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:56 smithi134 bash[18722]: audit 2024-02-10T08:20:55.381719+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:56.664 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:56 smithi134 bash[18722]: audit 2024-02-10T08:20:55.720143+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:56.665 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:56 smithi134 bash[18722]: audit 2024-02-10T08:20:55.731736+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[17385]: audit 2024-02-10T08:20:55.373095+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:20:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[17385]: cluster 2024-02-10T08:20:55.373234+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-10T08:20:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[17385]: audit 2024-02-10T08:20:55.374642+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[17385]: audit 2024-02-10T08:20:55.381719+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[17385]: audit 2024-02-10T08:20:55.720143+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[17385]: audit 2024-02-10T08:20:55.731736+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[24488]: audit 2024-02-10T08:20:55.373095+0000 mon.a (mon.0) 439 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:20:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[24488]: cluster 2024-02-10T08:20:55.373234+0000 mon.a (mon.0) 440 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-10T08:20:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[24488]: audit 2024-02-10T08:20:55.374642+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[24488]: audit 2024-02-10T08:20:55.381719+0000 mon.a (mon.0) 442 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[24488]: audit 2024-02-10T08:20:55.720143+0000 mon.a (mon.0) 443 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:20:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:56 smithi067 bash[24488]: audit 2024-02-10T08:20:55.731736+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: cluster 2024-02-10T08:20:54.618508+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: cluster 2024-02-10T08:20:54.618608+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: cluster 2024-02-10T08:20:56.385906+0000 mon.a (mon.0) 445 : cluster [INF] osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] boot 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: cluster 2024-02-10T08:20:56.386009+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: audit 2024-02-10T08:20:56.386881+0000 mon.a (mon.0) 447 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: cluster 2024-02-10T08:20:56.889685+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:57.761 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:57 smithi134 bash[18722]: cluster 2024-02-10T08:20:57.068794+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-10T08:20:57.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: cluster 2024-02-10T08:20:54.618508+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:20:57.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: cluster 2024-02-10T08:20:54.618608+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:20:57.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: cluster 2024-02-10T08:20:56.385906+0000 mon.a (mon.0) 445 : cluster [INF] osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] boot 2024-02-10T08:20:57.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: cluster 2024-02-10T08:20:56.386009+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-02-10T08:20:57.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: audit 2024-02-10T08:20:56.386881+0000 mon.a (mon.0) 447 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: cluster 2024-02-10T08:20:56.889685+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[24488]: cluster 2024-02-10T08:20:57.068794+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: cluster 2024-02-10T08:20:54.618508+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: cluster 2024-02-10T08:20:54.618608+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: cluster 2024-02-10T08:20:56.385906+0000 mon.a (mon.0) 445 : cluster [INF] osd.5 [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] boot 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: cluster 2024-02-10T08:20:56.386009+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: audit 2024-02-10T08:20:56.386881+0000 mon.a (mon.0) 447 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: cluster 2024-02-10T08:20:56.889685+0000 mgr.y (mgr.14146) 228 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-10T08:20:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:57 smithi067 bash[17385]: cluster 2024-02-10T08:20:57.068794+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-10T08:20:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:20:59 smithi134 bash[18722]: cluster 2024-02-10T08:20:58.078452+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-10T08:20:59.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:20:59 smithi067 bash[17385]: cluster 2024-02-10T08:20:58.078452+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-10T08:20:59.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:20:59 smithi067 bash[24488]: cluster 2024-02-10T08:20:58.078452+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-10T08:20:59.698 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-10T08:20:59.699 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-10T08:20:59.699 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-02-10T08:20:59.699 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-02-10T08:20:59.699 INFO:teuthology.orchestra.run.smithi134.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0164598 s, 637 MB/s 2024-02-10T08:20:59.699 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-02-10T08:21:00.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:00 smithi134 bash[18722]: cluster 2024-02-10T08:20:58.890231+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v215: 1 pgs: 1 remapped+peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:00 smithi067 bash[17385]: cluster 2024-02-10T08:20:58.890231+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v215: 1 pgs: 1 remapped+peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:00 smithi067 bash[24488]: cluster 2024-02-10T08:20:58.890231+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v215: 1 pgs: 1 remapped+peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:00.398 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-02-10T08:21:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:02 smithi067 bash[17385]: cluster 2024-02-10T08:21:00.890803+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v216: 1 pgs: 1 remapped+peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:02.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:02 smithi067 bash[24488]: cluster 2024-02-10T08:21:00.890803+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v216: 1 pgs: 1 remapped+peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:02 smithi134 bash[18722]: cluster 2024-02-10T08:21:00.890803+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v216: 1 pgs: 1 remapped+peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:03.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:03 smithi134 bash[18722]: cluster 2024-02-10T08:21:02.891413+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:03 smithi067 bash[17385]: cluster 2024-02-10T08:21:02.891413+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:03 smithi067 bash[24488]: cluster 2024-02-10T08:21:02.891413+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/44214941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f8620 msgr2=0x7f967c0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- 172.21.15.134:0/44214941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f8620 0x7f967c0fef20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9674009c10 tx=0x7f9674004d10).stop 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/44214941 shutdown_connections 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- 172.21.15.134:0/44214941 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f967c0f6a30 0x7f967c0f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- 172.21.15.134:0/44214941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f8620 0x7f967c0fef20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- 172.21.15.134:0/44214941 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f967c0f7cd0 0x7f967c0f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/44214941 >> 172.21.15.134:0/44214941 conn(0x7f967c0f2230 msgr2=0x7f967c0f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/44214941 shutdown_connections 2024-02-10T08:21:04.476 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/44214941 wait complete. 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 Processor -- start 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- start start 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f967c0f6a30 0x7f967c107b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 0x7f967c108000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f967c0f8620 0x7f967c10f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c10f5d0 con 0x7f967c0f6a30 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967c10f710 con 0x7f967c0f7cd0 2024-02-10T08:21:04.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967c10f850 con 0x7f967c0f8620 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 0x7f967c108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 0x7f967c108000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47830/0 (socket says 172.21.15.134:47830) 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 -- 172.21.15.134:0/2017972102 learned_addr learned my addr 172.21.15.134:0/2017972102 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967b7fe700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f967c0f6a30 0x7f967c107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967bfff700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f967c0f8620 0x7f967c10f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 -- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f967c0f8620 msgr2=0x7f967c10f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f967c0f8620 0x7f967c10f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 -- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f967c0f6a30 msgr2=0x7f967c107b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f967c0f6a30 0x7f967c107b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 -- 172.21.15.134:0/2017972102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96740097c0 con 0x7f967c0f7cd0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f967affd700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 0x7f967c108000 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f96700041d0 tx=0x7f9670009f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670007b40 con 0x7f967c0f7cd0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967c0f8f50 con 0x7f967c0f7cd0 2024-02-10T08:21:04.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f967000f040 con 0x7f967c0f7cd0 2024-02-10T08:21:04.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670014690 con 0x7f967c0f7cd0 2024-02-10T08:21:04.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.473+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f967c0f94c0 con 0x7f967c0f7cd0 2024-02-10T08:21:04.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.477+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9668004f50 con 0x7f967c0f7cd0 2024-02-10T08:21:04.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.477+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f9670009110 con 0x7f967c0f7cd0 2024-02-10T08:21:04.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.477+0000 7f9678ff9700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9664061250 0x7f9664063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:04.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.477+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4396+0+0 (secure 0 0 0) 0x7f967007c200 con 0x7f967c0f7cd0 2024-02-10T08:21:04.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.477+0000 7f967b7fe700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9664061250 0x7f9664063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:04.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.477+0000 7f967b7fe700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9664061250 0x7f9664063710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f967c0f7b30 tx=0x7f966c009040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:21:04.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.485+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f967004cb30 con 0x7f967c0f7cd0 2024-02-10T08:21:04.681 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:04.677+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9668000bf0 con 0x7f9664061250 2024-02-10T08:21:05.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:04 smithi134 bash[18722]: audit 2024-02-10T08:21:04.689910+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:21:05.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:04 smithi134 bash[18722]: audit 2024-02-10T08:21:04.696122+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:21:05.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:04 smithi134 bash[18722]: audit 2024-02-10T08:21:04.697432+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:04 smithi067 bash[17385]: audit 2024-02-10T08:21:04.689910+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:21:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:04 smithi067 bash[17385]: audit 2024-02-10T08:21:04.696122+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:21:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:04 smithi067 bash[17385]: audit 2024-02-10T08:21:04.697432+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:05.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:04 smithi067 bash[24488]: audit 2024-02-10T08:21:04.689910+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:21:05.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:04 smithi067 bash[24488]: audit 2024-02-10T08:21:04.696122+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:21:05.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:04 smithi067 bash[24488]: audit 2024-02-10T08:21:04.697432+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:05 smithi134 bash[18722]: audit 2024-02-10T08:21:04.682978+0000 mgr.y (mgr.14146) 232 : audit [DBG] from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:21:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:05 smithi134 bash[18722]: cluster 2024-02-10T08:21:04.891985+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:05 smithi067 bash[17385]: audit 2024-02-10T08:21:04.682978+0000 mgr.y (mgr.14146) 232 : audit [DBG] from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:21:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:05 smithi067 bash[17385]: cluster 2024-02-10T08:21:04.891985+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:06.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:05 smithi067 bash[24488]: audit 2024-02-10T08:21:04.682978+0000 mgr.y (mgr.14146) 232 : audit [DBG] from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:21:06.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:05 smithi067 bash[24488]: cluster 2024-02-10T08:21:04.891985+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:07.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: cephadm 2024-02-10T08:21:06.106545+0000 mgr.y (mgr.14146) 234 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.116488+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.118227+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.119598+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: cephadm 2024-02-10T08:21:06.120647+0000 mgr.y (mgr.14146) 235 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 8632M 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.125539+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.127440+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.128749+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: audit 2024-02-10T08:21:06.135183+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[24488]: cluster 2024-02-10T08:21:06.892526+0000 mgr.y (mgr.14146) 236 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: cephadm 2024-02-10T08:21:06.106545+0000 mgr.y (mgr.14146) 234 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:21:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.116488+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.118227+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.119598+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: cephadm 2024-02-10T08:21:06.120647+0000 mgr.y (mgr.14146) 235 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 8632M 2024-02-10T08:21:07.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.125539+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.127440+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:07.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.128749+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:07.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: audit 2024-02-10T08:21:06.135183+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:07 smithi067 bash[17385]: cluster 2024-02-10T08:21:06.892526+0000 mgr.y (mgr.14146) 236 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:07.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: cephadm 2024-02-10T08:21:06.106545+0000 mgr.y (mgr.14146) 234 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:21:07.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.116488+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.118227+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.119598+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: cephadm 2024-02-10T08:21:06.120647+0000 mgr.y (mgr.14146) 235 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 8632M 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.125539+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.127440+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.128749+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: audit 2024-02-10T08:21:06.135183+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:07 smithi134 bash[18722]: cluster 2024-02-10T08:21:06.892526+0000 mgr.y (mgr.14146) 236 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:10.228 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:09 smithi134 bash[18722]: cluster 2024-02-10T08:21:08.893088+0000 mgr.y (mgr.14146) 237 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:09 smithi067 bash[17385]: cluster 2024-02-10T08:21:08.893088+0000 mgr.y (mgr.14146) 237 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:09 smithi067 bash[24488]: cluster 2024-02-10T08:21:08.893088+0000 mgr.y (mgr.14146) 237 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:11 smithi134 bash[18722]: cluster 2024-02-10T08:21:10.893681+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:11 smithi067 bash[17385]: cluster 2024-02-10T08:21:10.893681+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:11 smithi067 bash[24488]: cluster 2024-02-10T08:21:10.893681+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[17385]: audit 2024-02-10T08:21:12.621963+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.134:0/3678172492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]: dispatch 2024-02-10T08:21:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[17385]: audit 2024-02-10T08:21:12.623136+0000 mon.a (mon.0) 460 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]: dispatch 2024-02-10T08:21:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[17385]: audit 2024-02-10T08:21:12.631829+0000 mon.a (mon.0) 461 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]': finished 2024-02-10T08:21:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[17385]: cluster 2024-02-10T08:21:12.632001+0000 mon.a (mon.0) 462 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-02-10T08:21:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[17385]: audit 2024-02-10T08:21:12.632248+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[24488]: audit 2024-02-10T08:21:12.621963+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.134:0/3678172492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]: dispatch 2024-02-10T08:21:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[24488]: audit 2024-02-10T08:21:12.623136+0000 mon.a (mon.0) 460 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]: dispatch 2024-02-10T08:21:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[24488]: audit 2024-02-10T08:21:12.631829+0000 mon.a (mon.0) 461 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]': finished 2024-02-10T08:21:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[24488]: cluster 2024-02-10T08:21:12.632001+0000 mon.a (mon.0) 462 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-02-10T08:21:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:13 smithi067 bash[24488]: audit 2024-02-10T08:21:12.632248+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:13 smithi134 bash[18722]: audit 2024-02-10T08:21:12.621963+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.134:0/3678172492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]: dispatch 2024-02-10T08:21:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:13 smithi134 bash[18722]: audit 2024-02-10T08:21:12.623136+0000 mon.a (mon.0) 460 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]: dispatch 2024-02-10T08:21:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:13 smithi134 bash[18722]: audit 2024-02-10T08:21:12.631829+0000 mon.a (mon.0) 461 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92ea5976-343a-4e4c-8b06-79ee579d4a84"}]': finished 2024-02-10T08:21:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:13 smithi134 bash[18722]: cluster 2024-02-10T08:21:12.632001+0000 mon.a (mon.0) 462 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-02-10T08:21:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:13 smithi134 bash[18722]: audit 2024-02-10T08:21:12.632248+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:14 smithi067 bash[17385]: cluster 2024-02-10T08:21:12.894260+0000 mgr.y (mgr.14146) 239 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:14 smithi067 bash[17385]: audit 2024-02-10T08:21:13.353067+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.134:0/1228667703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:21:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:14 smithi067 bash[24488]: cluster 2024-02-10T08:21:12.894260+0000 mgr.y (mgr.14146) 239 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:14.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:14 smithi067 bash[24488]: audit 2024-02-10T08:21:13.353067+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.134:0/1228667703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:21:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:14 smithi134 bash[18722]: cluster 2024-02-10T08:21:12.894260+0000 mgr.y (mgr.14146) 239 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:14 smithi134 bash[18722]: audit 2024-02-10T08:21:13.353067+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.134:0/1228667703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:21:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:16 smithi134 bash[18722]: cluster 2024-02-10T08:21:14.894877+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:16 smithi067 bash[17385]: cluster 2024-02-10T08:21:14.894877+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:16.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:16 smithi067 bash[24488]: cluster 2024-02-10T08:21:14.894877+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:18.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:18 smithi067 bash[17385]: cluster 2024-02-10T08:21:16.895418+0000 mgr.y (mgr.14146) 241 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:18.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:18 smithi067 bash[24488]: cluster 2024-02-10T08:21:16.895418+0000 mgr.y (mgr.14146) 241 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:18 smithi134 bash[18722]: cluster 2024-02-10T08:21:16.895418+0000 mgr.y (mgr.14146) 241 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:20 smithi067 bash[17385]: cluster 2024-02-10T08:21:18.895985+0000 mgr.y (mgr.14146) 242 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:20.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:20 smithi067 bash[24488]: cluster 2024-02-10T08:21:18.895985+0000 mgr.y (mgr.14146) 242 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:20 smithi134 bash[18722]: cluster 2024-02-10T08:21:18.895985+0000 mgr.y (mgr.14146) 242 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:22.362 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:22 smithi134 bash[18722]: cluster 2024-02-10T08:21:20.896573+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:22 smithi067 bash[17385]: cluster 2024-02-10T08:21:20.896573+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:22.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:22 smithi067 bash[24488]: cluster 2024-02-10T08:21:20.896573+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:23 smithi134 bash[18722]: cluster 2024-02-10T08:21:22.897120+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:23 smithi067 bash[17385]: cluster 2024-02-10T08:21:22.897120+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:23 smithi067 bash[24488]: cluster 2024-02-10T08:21:22.897120+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:26.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:25 smithi134 bash[18722]: cluster 2024-02-10T08:21:24.897672+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:26.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:25 smithi134 bash[18722]: audit 2024-02-10T08:21:24.978583+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T08:21:26.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:25 smithi134 bash[18722]: audit 2024-02-10T08:21:24.980275+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:26.240 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:25 smithi134 bash[18722]: cephadm 2024-02-10T08:21:24.981924+0000 mgr.y (mgr.14146) 246 : cephadm [INF] Deploying daemon osd.6 on smithi134 2024-02-10T08:21:26.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[24488]: cluster 2024-02-10T08:21:24.897672+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:26.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[24488]: audit 2024-02-10T08:21:24.978583+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T08:21:26.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[24488]: audit 2024-02-10T08:21:24.980275+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:26.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[24488]: cephadm 2024-02-10T08:21:24.981924+0000 mgr.y (mgr.14146) 246 : cephadm [INF] Deploying daemon osd.6 on smithi134 2024-02-10T08:21:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[17385]: cluster 2024-02-10T08:21:24.897672+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[17385]: audit 2024-02-10T08:21:24.978583+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-10T08:21:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[17385]: audit 2024-02-10T08:21:24.980275+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:25 smithi067 bash[17385]: cephadm 2024-02-10T08:21:24.981924+0000 mgr.y (mgr.14146) 246 : cephadm [INF] Deploying daemon osd.6 on smithi134 2024-02-10T08:21:28.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:28 smithi134 bash[18722]: cluster 2024-02-10T08:21:26.898361+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:28.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:28 smithi067 bash[24488]: cluster 2024-02-10T08:21:26.898361+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:28.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:28 smithi067 bash[17385]: cluster 2024-02-10T08:21:26.898361+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:30.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:30 smithi134 bash[18722]: cluster 2024-02-10T08:21:28.899105+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:30.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:30 smithi134 bash[18722]: audit 2024-02-10T08:21:29.048672+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:30.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:30 smithi134 bash[18722]: audit 2024-02-10T08:21:29.064809+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:21:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:30 smithi067 bash[17385]: cluster 2024-02-10T08:21:28.899105+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:30 smithi067 bash[17385]: audit 2024-02-10T08:21:29.048672+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:30 smithi067 bash[17385]: audit 2024-02-10T08:21:29.064809+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:21:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:30 smithi067 bash[24488]: cluster 2024-02-10T08:21:28.899105+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:30 smithi067 bash[24488]: audit 2024-02-10T08:21:29.048672+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:30 smithi067 bash[24488]: audit 2024-02-10T08:21:29.064809+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:21:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:32 smithi067 bash[17385]: cluster 2024-02-10T08:21:30.899693+0000 mgr.y (mgr.14146) 249 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:32 smithi067 bash[24488]: cluster 2024-02-10T08:21:30.899693+0000 mgr.y (mgr.14146) 249 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:32.414 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:32 smithi134 bash[18722]: cluster 2024-02-10T08:21:30.899693+0000 mgr.y (mgr.14146) 249 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:34 smithi067 bash[17385]: cluster 2024-02-10T08:21:32.900412+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:34 smithi067 bash[24488]: cluster 2024-02-10T08:21:32.900412+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:34 smithi134 bash[18722]: cluster 2024-02-10T08:21:32.900412+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:35 smithi134 bash[18722]: audit 2024-02-10T08:21:34.213372+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:35 smithi134 bash[18722]: audit 2024-02-10T08:21:34.506861+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:35 smithi134 bash[18722]: audit 2024-02-10T08:21:34.508837+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:35 smithi134 bash[18722]: audit 2024-02-10T08:21:34.517456+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:35 smithi134 bash[18722]: cluster 2024-02-10T08:21:34.901072+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[17385]: audit 2024-02-10T08:21:34.213372+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[17385]: audit 2024-02-10T08:21:34.506861+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[17385]: audit 2024-02-10T08:21:34.508837+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[17385]: audit 2024-02-10T08:21:34.517456+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[17385]: cluster 2024-02-10T08:21:34.901072+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[24488]: audit 2024-02-10T08:21:34.213372+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[24488]: audit 2024-02-10T08:21:34.506861+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[24488]: audit 2024-02-10T08:21:34.508837+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[24488]: audit 2024-02-10T08:21:34.517456+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:35 smithi067 bash[24488]: cluster 2024-02-10T08:21:34.901072+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:36.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:36 smithi134 bash[18722]: audit 2024-02-10T08:21:35.447507+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T08:21:36.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:36 smithi134 bash[18722]: audit 2024-02-10T08:21:35.448395+0000 mon.a (mon.0) 472 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T08:21:36.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:36 smithi067 bash[17385]: audit 2024-02-10T08:21:35.447507+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T08:21:36.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:36 smithi067 bash[17385]: audit 2024-02-10T08:21:35.448395+0000 mon.a (mon.0) 472 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T08:21:36.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:36 smithi067 bash[24488]: audit 2024-02-10T08:21:35.447507+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T08:21:36.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:36 smithi067 bash[24488]: audit 2024-02-10T08:21:35.448395+0000 mon.a (mon.0) 472 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-10T08:21:37.475 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.468+0000 7f9678ff9700 1 -- 172.21.15.134:0/2017972102 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9668000bf0 con 0x7f9664061250 2024-02-10T08:21:37.475 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 6 on host 'smithi134' 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9664061250 msgr2=0x7f9664063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9664061250 0x7f9664063710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f967c0f7b30 tx=0x7f966c009040).stop 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 msgr2=0x7f967c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 0x7f967c108000 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f96700041d0 tx=0x7f9670009f60).stop 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 shutdown_connections 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f9664061250 0x7f9664063710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:37.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f967c0f6a30 0x7f967c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:37.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f967c0f8620 0x7f967c10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:37.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 --2- 172.21.15.134:0/2017972102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f967c0f7cd0 0x7f967c108000 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:37.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 >> 172.21.15.134:0/2017972102 conn(0x7f967c0f2230 msgr2=0x7f967c060a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:21:37.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 shutdown_connections 2024-02-10T08:21:37.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:37.472+0000 7f96821b1700 1 -- 172.21.15.134:0/2017972102 wait complete. 2024-02-10T08:21:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:37 smithi134 bash[18722]: audit 2024-02-10T08:21:36.224109+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T08:21:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:37 smithi134 bash[18722]: cluster 2024-02-10T08:21:36.224177+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-10T08:21:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:37 smithi134 bash[18722]: audit 2024-02-10T08:21:36.224318+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:37 smithi134 bash[18722]: audit 2024-02-10T08:21:36.224889+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:21:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:37 smithi134 bash[18722]: audit 2024-02-10T08:21:36.225506+0000 mon.a (mon.0) 476 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:21:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:37 smithi134 bash[18722]: cluster 2024-02-10T08:21:36.901628+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[17385]: audit 2024-02-10T08:21:36.224109+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T08:21:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[17385]: cluster 2024-02-10T08:21:36.224177+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-10T08:21:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[17385]: audit 2024-02-10T08:21:36.224318+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[17385]: audit 2024-02-10T08:21:36.224889+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[17385]: audit 2024-02-10T08:21:36.225506+0000 mon.a (mon.0) 476 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[17385]: cluster 2024-02-10T08:21:36.901628+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[24488]: audit 2024-02-10T08:21:36.224109+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[24488]: cluster 2024-02-10T08:21:36.224177+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[24488]: audit 2024-02-10T08:21:36.224318+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[24488]: audit 2024-02-10T08:21:36.224889+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[24488]: audit 2024-02-10T08:21:36.225506+0000 mon.a (mon.0) 476 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:21:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:37 smithi067 bash[24488]: cluster 2024-02-10T08:21:36.901628+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:38.223 DEBUG:teuthology.orchestra.run.smithi134:osd.6> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.6.service 2024-02-10T08:21:38.226 INFO:tasks.cephadm:Deploying osd.7 on smithi134 with /dev/vg_nvme/lv_1... 2024-02-10T08:21:38.227 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-10T08:21:38.458 INFO:journalctl@ceph.osd.6.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:21:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:38 smithi134 bash[18722]: audit 2024-02-10T08:21:37.228948+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:21:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:38 smithi134 bash[18722]: cluster 2024-02-10T08:21:37.229106+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-10T08:21:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:38 smithi134 bash[18722]: audit 2024-02-10T08:21:37.229702+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:38 smithi134 bash[18722]: audit 2024-02-10T08:21:37.235633+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:38 smithi134 bash[18722]: audit 2024-02-10T08:21:37.472888+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:38 smithi134 bash[18722]: audit 2024-02-10T08:21:37.494201+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:21:38.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[24488]: audit 2024-02-10T08:21:37.228948+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:21:38.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[24488]: cluster 2024-02-10T08:21:37.229106+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-10T08:21:38.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[24488]: audit 2024-02-10T08:21:37.229702+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:38.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[24488]: audit 2024-02-10T08:21:37.235633+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:38.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[24488]: audit 2024-02-10T08:21:37.472888+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[24488]: audit 2024-02-10T08:21:37.494201+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[17385]: audit 2024-02-10T08:21:37.228948+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[17385]: cluster 2024-02-10T08:21:37.229106+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[17385]: audit 2024-02-10T08:21:37.229702+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[17385]: audit 2024-02-10T08:21:37.235633+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[17385]: audit 2024-02-10T08:21:37.472888+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:38 smithi067 bash[17385]: audit 2024-02-10T08:21:37.494201+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:21:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: cluster 2024-02-10T08:21:36.412217+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:21:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: cluster 2024-02-10T08:21:36.412283+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:21:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: audit 2024-02-10T08:21:38.241696+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: cluster 2024-02-10T08:21:38.246425+0000 mon.a (mon.0) 484 : cluster [INF] osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] boot 2024-02-10T08:21:39.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: cluster 2024-02-10T08:21:38.246492+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-02-10T08:21:39.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: audit 2024-02-10T08:21:38.246731+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:39.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:39 smithi134 bash[18722]: cluster 2024-02-10T08:21:38.902169+0000 mgr.y (mgr.14146) 253 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: cluster 2024-02-10T08:21:36.412217+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:21:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: cluster 2024-02-10T08:21:36.412283+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:21:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: audit 2024-02-10T08:21:38.241696+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: cluster 2024-02-10T08:21:38.246425+0000 mon.a (mon.0) 484 : cluster [INF] osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] boot 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: cluster 2024-02-10T08:21:38.246492+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: audit 2024-02-10T08:21:38.246731+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[17385]: cluster 2024-02-10T08:21:38.902169+0000 mgr.y (mgr.14146) 253 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: cluster 2024-02-10T08:21:36.412217+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: cluster 2024-02-10T08:21:36.412283+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: audit 2024-02-10T08:21:38.241696+0000 mon.a (mon.0) 483 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: cluster 2024-02-10T08:21:38.246425+0000 mon.a (mon.0) 484 : cluster [INF] osd.6 [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] boot 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: cluster 2024-02-10T08:21:38.246492+0000 mon.a (mon.0) 485 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: audit 2024-02-10T08:21:38.246731+0000 mon.a (mon.0) 486 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:21:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:39 smithi067 bash[24488]: cluster 2024-02-10T08:21:38.902169+0000 mgr.y (mgr.14146) 253 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-10T08:21:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:40 smithi134 bash[18722]: cluster 2024-02-10T08:21:39.251715+0000 mon.a (mon.0) 487 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-10T08:21:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:40 smithi067 bash[17385]: cluster 2024-02-10T08:21:39.251715+0000 mon.a (mon.0) 487 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-10T08:21:40.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:40 smithi067 bash[24488]: cluster 2024-02-10T08:21:39.251715+0000 mon.a (mon.0) 487 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-10T08:21:41.459 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-10T08:21:41.459 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-10T08:21:41.459 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-02-10T08:21:41.459 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-02-10T08:21:41.459 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0172449 s, 608 MB/s 2024-02-10T08:21:41.459 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-02-10T08:21:41.520 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:41 smithi134 bash[18722]: cluster 2024-02-10T08:21:40.260231+0000 mon.a (mon.0) 488 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-10T08:21:41.521 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:41 smithi134 bash[18722]: cluster 2024-02-10T08:21:40.902693+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:41 smithi067 bash[17385]: cluster 2024-02-10T08:21:40.260231+0000 mon.a (mon.0) 488 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-10T08:21:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:41 smithi067 bash[17385]: cluster 2024-02-10T08:21:40.902693+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:41 smithi067 bash[24488]: cluster 2024-02-10T08:21:40.260231+0000 mon.a (mon.0) 488 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-10T08:21:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:41 smithi067 bash[24488]: cluster 2024-02-10T08:21:40.902693+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:42.194 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-02-10T08:21:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:44 smithi134 bash[18722]: cluster 2024-02-10T08:21:42.903268+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:44 smithi067 bash[17385]: cluster 2024-02-10T08:21:42.903268+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:44 smithi067 bash[24488]: cluster 2024-02-10T08:21:42.903268+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:46 smithi067 bash[17385]: cluster 2024-02-10T08:21:44.903837+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:46 smithi067 bash[24488]: cluster 2024-02-10T08:21:44.903837+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/1773480709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 msgr2=0x7ff718102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/1773480709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718102220 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7ff710009c10 tx=0x7ff710004d10).stop 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/1773480709 shutdown_connections 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/1773480709 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7180f9d30 0x7ff7180fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/1773480709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718102220 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/1773480709 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ff7180fafd0 0x7ff7180fb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:46.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/1773480709 >> 172.21.15.134:0/1773480709 conn(0x7ff7180f5710 msgr2=0x7ff7180f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:21:46.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/1773480709 shutdown_connections 2024-02-10T08:21:46.385 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/1773480709 wait complete. 2024-02-10T08:21:46.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 Processor -- start 2024-02-10T08:21:46.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- start start 2024-02-10T08:21:46.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ff7180f9d30 0x7ff718109da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:46.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7180fafd0 0x7ff71810a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:46.386 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718111330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff718111820 con 0x7ff7180fafd0 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff718111960 con 0x7ff7180fb920 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff718111af0 con 0x7ff7180f9d30 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718111330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718111330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:57612/0 (socket says 172.21.15.134:57612) 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 -- 172.21.15.134:0/3667369026 learned_addr learned my addr 172.21.15.134:0/3667369026 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71cd7d700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7180fafd0 0x7ff71810a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71d57e700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ff7180f9d30 0x7ff718109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 -- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ff7180f9d30 msgr2=0x7ff718109da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ff7180f9d30 0x7ff718109da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 -- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7180fafd0 msgr2=0x7ff71810a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7180fafd0 0x7ff71810a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:21:46.387 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 -- 172.21.15.134:0/3667369026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100097c0 con 0x7ff7180fb920 2024-02-10T08:21:46.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71dd7f700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718111330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff71000b890 tx=0x7ff710004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:21:46.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71000e040 con 0x7ff7180fb920 2024-02-10T08:21:46.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff718111d70 con 0x7ff7180fb920 2024-02-10T08:21:46.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7ff710018040 con 0x7ff7180fb920 2024-02-10T08:21:46.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71001d400 con 0x7ff7180fb920 2024-02-10T08:21:46.388 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.380+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff718112180 con 0x7ff7180fb920 2024-02-10T08:21:46.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.384+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7181048b0 con 0x7ff7180fb920 2024-02-10T08:21:46.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.384+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7ff710013420 con 0x7ff7180fb920 2024-02-10T08:21:46.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.384+0000 7ff70e7fc700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff704061280 0x7ff704063740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:21:46.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.384+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4828+0+0 (secure 0 0 0) 0x7ff71007df80 con 0x7ff7180fb920 2024-02-10T08:21:46.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.384+0000 7ff71d57e700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff704061280 0x7ff704063740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:21:46.393 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.388+0000 7ff71d57e700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff704061280 0x7ff704063740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff71810b290 tx=0x7ff714009e00).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:21:46.394 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.388+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7ff71004e7a0 con 0x7ff7180fb920 2024-02-10T08:21:46.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:46 smithi134 bash[18722]: cluster 2024-02-10T08:21:44.903837+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:46.585 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:21:46.580+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff718101040 con 0x7ff704061280 2024-02-10T08:21:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:47 smithi067 bash[17385]: audit 2024-02-10T08:21:46.593581+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:21:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:47 smithi067 bash[17385]: audit 2024-02-10T08:21:46.598885+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:21:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:47 smithi067 bash[17385]: audit 2024-02-10T08:21:46.600392+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:47 smithi067 bash[24488]: audit 2024-02-10T08:21:46.593581+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:21:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:47 smithi067 bash[24488]: audit 2024-02-10T08:21:46.598885+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:21:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:47 smithi067 bash[24488]: audit 2024-02-10T08:21:46.600392+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:47.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:47 smithi134 bash[18722]: audit 2024-02-10T08:21:46.593581+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T08:21:47.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:47 smithi134 bash[18722]: audit 2024-02-10T08:21:46.598885+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T08:21:47.403 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:47 smithi134 bash[18722]: audit 2024-02-10T08:21:46.600392+0000 mon.a (mon.0) 491 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:48 smithi067 bash[17385]: audit 2024-02-10T08:21:46.586649+0000 mgr.y (mgr.14146) 257 : audit [DBG] from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:21:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:48 smithi067 bash[17385]: cluster 2024-02-10T08:21:46.904400+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:48.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:48 smithi067 bash[24488]: audit 2024-02-10T08:21:46.586649+0000 mgr.y (mgr.14146) 257 : audit [DBG] from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:21:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:48 smithi067 bash[24488]: cluster 2024-02-10T08:21:46.904400+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:48 smithi134 bash[18722]: audit 2024-02-10T08:21:46.586649+0000 mgr.y (mgr.14146) 257 : audit [DBG] from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:21:48.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:48 smithi134 bash[18722]: cluster 2024-02-10T08:21:46.904400+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: cephadm 2024-02-10T08:21:48.190530+0000 mgr.y (mgr.14146) 259 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:21:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.200062+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.202391+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.203646+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.204825+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: cephadm 2024-02-10T08:21:48.205658+0000 mgr.y (mgr.14146) 260 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 5755M 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.213467+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.215152+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.216363+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: audit 2024-02-10T08:21:48.225977+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:49 smithi134 bash[18722]: cluster 2024-02-10T08:21:48.905025+0000 mgr.y (mgr.14146) 261 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: cephadm 2024-02-10T08:21:48.190530+0000 mgr.y (mgr.14146) 259 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:21:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.200062+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.202391+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.203646+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.204825+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: cephadm 2024-02-10T08:21:48.205658+0000 mgr.y (mgr.14146) 260 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 5755M 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.213467+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.215152+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.216363+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: audit 2024-02-10T08:21:48.225977+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[17385]: cluster 2024-02-10T08:21:48.905025+0000 mgr.y (mgr.14146) 261 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: cephadm 2024-02-10T08:21:48.190530+0000 mgr.y (mgr.14146) 259 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:21:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.200062+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.202391+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.203646+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.204825+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: cephadm 2024-02-10T08:21:48.205658+0000 mgr.y (mgr.14146) 260 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 5755M 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.213467+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.215152+0000 mon.a (mon.0) 497 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.216363+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: audit 2024-02-10T08:21:48.225977+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:21:49.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:49 smithi067 bash[24488]: cluster 2024-02-10T08:21:48.905025+0000 mgr.y (mgr.14146) 261 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:52.255 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:51 smithi134 bash[18722]: cluster 2024-02-10T08:21:50.905646+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:51 smithi067 bash[17385]: cluster 2024-02-10T08:21:50.905646+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:51 smithi067 bash[24488]: cluster 2024-02-10T08:21:50.905646+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:54.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:54 smithi067 bash[17385]: cluster 2024-02-10T08:21:52.906238+0000 mgr.y (mgr.14146) 263 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:54.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:54 smithi067 bash[24488]: cluster 2024-02-10T08:21:52.906238+0000 mgr.y (mgr.14146) 263 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:54 smithi134 bash[18722]: cluster 2024-02-10T08:21:52.906238+0000 mgr.y (mgr.14146) 263 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:55.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[17385]: audit 2024-02-10T08:21:54.658241+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.134:0/899597310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]: dispatch 2024-02-10T08:21:55.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[17385]: audit 2024-02-10T08:21:54.659548+0000 mon.a (mon.0) 500 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]: dispatch 2024-02-10T08:21:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[17385]: audit 2024-02-10T08:21:54.668634+0000 mon.a (mon.0) 501 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]': finished 2024-02-10T08:21:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[17385]: cluster 2024-02-10T08:21:54.668810+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-02-10T08:21:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[17385]: audit 2024-02-10T08:21:54.669048+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:21:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[24488]: audit 2024-02-10T08:21:54.658241+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.134:0/899597310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]: dispatch 2024-02-10T08:21:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[24488]: audit 2024-02-10T08:21:54.659548+0000 mon.a (mon.0) 500 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]: dispatch 2024-02-10T08:21:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[24488]: audit 2024-02-10T08:21:54.668634+0000 mon.a (mon.0) 501 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]': finished 2024-02-10T08:21:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[24488]: cluster 2024-02-10T08:21:54.668810+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-02-10T08:21:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:55 smithi067 bash[24488]: audit 2024-02-10T08:21:54.669048+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:21:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:55 smithi134 bash[18722]: audit 2024-02-10T08:21:54.658241+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.134:0/899597310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]: dispatch 2024-02-10T08:21:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:55 smithi134 bash[18722]: audit 2024-02-10T08:21:54.659548+0000 mon.a (mon.0) 500 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]: dispatch 2024-02-10T08:21:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:55 smithi134 bash[18722]: audit 2024-02-10T08:21:54.668634+0000 mon.a (mon.0) 501 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4ebe0339-e352-47e1-adba-42e1cd076b46"}]': finished 2024-02-10T08:21:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:55 smithi134 bash[18722]: cluster 2024-02-10T08:21:54.668810+0000 mon.a (mon.0) 502 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-02-10T08:21:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:55 smithi134 bash[18722]: audit 2024-02-10T08:21:54.669048+0000 mon.a (mon.0) 503 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:21:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:56 smithi067 bash[17385]: cluster 2024-02-10T08:21:54.906813+0000 mgr.y (mgr.14146) 264 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:56 smithi067 bash[17385]: audit 2024-02-10T08:21:55.380426+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.134:0/2377479568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:21:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:56 smithi067 bash[24488]: cluster 2024-02-10T08:21:54.906813+0000 mgr.y (mgr.14146) 264 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:56 smithi067 bash[24488]: audit 2024-02-10T08:21:55.380426+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.134:0/2377479568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:21:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:56 smithi134 bash[18722]: cluster 2024-02-10T08:21:54.906813+0000 mgr.y (mgr.14146) 264 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:56 smithi134 bash[18722]: audit 2024-02-10T08:21:55.380426+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.134:0/2377479568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T08:21:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:21:58 smithi134 bash[18722]: cluster 2024-02-10T08:21:56.907379+0000 mgr.y (mgr.14146) 265 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:58.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:21:58 smithi067 bash[17385]: cluster 2024-02-10T08:21:56.907379+0000 mgr.y (mgr.14146) 265 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:21:58.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:21:58 smithi067 bash[24488]: cluster 2024-02-10T08:21:56.907379+0000 mgr.y (mgr.14146) 265 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:00 smithi067 bash[17385]: cluster 2024-02-10T08:21:58.907959+0000 mgr.y (mgr.14146) 266 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:00 smithi067 bash[24488]: cluster 2024-02-10T08:21:58.907959+0000 mgr.y (mgr.14146) 266 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:00 smithi134 bash[18722]: cluster 2024-02-10T08:21:58.907959+0000 mgr.y (mgr.14146) 266 : cluster [DBG] pgmap v252: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:02 smithi067 bash[17385]: cluster 2024-02-10T08:22:00.908523+0000 mgr.y (mgr.14146) 267 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:02.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:02 smithi067 bash[24488]: cluster 2024-02-10T08:22:00.908523+0000 mgr.y (mgr.14146) 267 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:02 smithi134 bash[18722]: cluster 2024-02-10T08:22:00.908523+0000 mgr.y (mgr.14146) 267 : cluster [DBG] pgmap v253: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:04.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:04 smithi067 bash[17385]: cluster 2024-02-10T08:22:02.909073+0000 mgr.y (mgr.14146) 268 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:04.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:04 smithi067 bash[24488]: cluster 2024-02-10T08:22:02.909073+0000 mgr.y (mgr.14146) 268 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:04.454 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:04 smithi134 bash[18722]: cluster 2024-02-10T08:22:02.909073+0000 mgr.y (mgr.14146) 268 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:06 smithi067 bash[17385]: cluster 2024-02-10T08:22:04.909617+0000 mgr.y (mgr.14146) 269 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:06 smithi067 bash[24488]: cluster 2024-02-10T08:22:04.909617+0000 mgr.y (mgr.14146) 269 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:06.381 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:06 smithi134 bash[18722]: cluster 2024-02-10T08:22:04.909617+0000 mgr.y (mgr.14146) 269 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[17385]: cluster 2024-02-10T08:22:06.910195+0000 mgr.y (mgr.14146) 270 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[17385]: audit 2024-02-10T08:22:06.994202+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T08:22:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[17385]: audit 2024-02-10T08:22:06.996086+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[17385]: cephadm 2024-02-10T08:22:06.997620+0000 mgr.y (mgr.14146) 271 : cephadm [INF] Deploying daemon osd.7 on smithi134 2024-02-10T08:22:07.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[24488]: cluster 2024-02-10T08:22:06.910195+0000 mgr.y (mgr.14146) 270 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:07.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[24488]: audit 2024-02-10T08:22:06.994202+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T08:22:07.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[24488]: audit 2024-02-10T08:22:06.996086+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:07.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:07 smithi067 bash[24488]: cephadm 2024-02-10T08:22:06.997620+0000 mgr.y (mgr.14146) 271 : cephadm [INF] Deploying daemon osd.7 on smithi134 2024-02-10T08:22:07.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:07 smithi134 bash[18722]: cluster 2024-02-10T08:22:06.910195+0000 mgr.y (mgr.14146) 270 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:07 smithi134 bash[18722]: audit 2024-02-10T08:22:06.994202+0000 mon.a (mon.0) 504 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-10T08:22:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:07 smithi134 bash[18722]: audit 2024-02-10T08:22:06.996086+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:07.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:07 smithi134 bash[18722]: cephadm 2024-02-10T08:22:06.997620+0000 mgr.y (mgr.14146) 271 : cephadm [INF] Deploying daemon osd.7 on smithi134 2024-02-10T08:22:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:09 smithi134 bash[18722]: cluster 2024-02-10T08:22:08.910735+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:10.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:09 smithi067 bash[17385]: cluster 2024-02-10T08:22:08.910735+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:10.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:09 smithi067 bash[24488]: cluster 2024-02-10T08:22:08.910735+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:12 smithi134 bash[18722]: cluster 2024-02-10T08:22:10.911272+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:12 smithi134 bash[18722]: audit 2024-02-10T08:22:11.022415+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:12 smithi134 bash[18722]: audit 2024-02-10T08:22:11.059306+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:12 smithi067 bash[17385]: cluster 2024-02-10T08:22:10.911272+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:12 smithi067 bash[17385]: audit 2024-02-10T08:22:11.022415+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:12 smithi067 bash[17385]: audit 2024-02-10T08:22:11.059306+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:12 smithi067 bash[24488]: cluster 2024-02-10T08:22:10.911272+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v258: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:12 smithi067 bash[24488]: audit 2024-02-10T08:22:11.022415+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:12 smithi067 bash[24488]: audit 2024-02-10T08:22:11.059306+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:14 smithi067 bash[17385]: cluster 2024-02-10T08:22:12.911823+0000 mgr.y (mgr.14146) 274 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:14 smithi067 bash[24488]: cluster 2024-02-10T08:22:12.911823+0000 mgr.y (mgr.14146) 274 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:14.451 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:14 smithi134 bash[18722]: cluster 2024-02-10T08:22:12.911823+0000 mgr.y (mgr.14146) 274 : cluster [DBG] pgmap v259: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:16.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:16 smithi067 bash[17385]: cluster 2024-02-10T08:22:14.912384+0000 mgr.y (mgr.14146) 275 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:16.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:16 smithi067 bash[24488]: cluster 2024-02-10T08:22:14.912384+0000 mgr.y (mgr.14146) 275 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:16 smithi134 bash[18722]: cluster 2024-02-10T08:22:14.912384+0000 mgr.y (mgr.14146) 275 : cluster [DBG] pgmap v260: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:17.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:17 smithi134 bash[18722]: audit 2024-02-10T08:22:16.217608+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:17.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:17 smithi134 bash[18722]: audit 2024-02-10T08:22:16.219130+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:17.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:17 smithi134 bash[18722]: audit 2024-02-10T08:22:16.220318+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:17.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:17 smithi134 bash[18722]: audit 2024-02-10T08:22:16.227031+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:17.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:17 smithi134 bash[18722]: cluster 2024-02-10T08:22:16.912922+0000 mgr.y (mgr.14146) 276 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[17385]: audit 2024-02-10T08:22:16.217608+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[17385]: audit 2024-02-10T08:22:16.219130+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[17385]: audit 2024-02-10T08:22:16.220318+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[17385]: audit 2024-02-10T08:22:16.227031+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[17385]: cluster 2024-02-10T08:22:16.912922+0000 mgr.y (mgr.14146) 276 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[24488]: audit 2024-02-10T08:22:16.217608+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[24488]: audit 2024-02-10T08:22:16.219130+0000 mon.a (mon.0) 509 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[24488]: audit 2024-02-10T08:22:16.220318+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[24488]: audit 2024-02-10T08:22:16.227031+0000 mon.a (mon.0) 511 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:17 smithi067 bash[24488]: cluster 2024-02-10T08:22:16.912922+0000 mgr.y (mgr.14146) 276 : cluster [DBG] pgmap v261: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:18 smithi134 bash[18722]: audit 2024-02-10T08:22:17.465234+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T08:22:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:18 smithi134 bash[18722]: audit 2024-02-10T08:22:17.466110+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T08:22:18.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:18 smithi067 bash[17385]: audit 2024-02-10T08:22:17.465234+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T08:22:18.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:18 smithi067 bash[17385]: audit 2024-02-10T08:22:17.466110+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T08:22:18.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:18 smithi067 bash[24488]: audit 2024-02-10T08:22:17.465234+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T08:22:18.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:18 smithi067 bash[24488]: audit 2024-02-10T08:22:17.466110+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-10T08:22:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:19 smithi134 bash[18722]: audit 2024-02-10T08:22:18.240603+0000 mon.a (mon.0) 513 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T08:22:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:19 smithi134 bash[18722]: cluster 2024-02-10T08:22:18.240718+0000 mon.a (mon.0) 514 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-10T08:22:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:19 smithi134 bash[18722]: audit 2024-02-10T08:22:18.240913+0000 mon.a (mon.0) 515 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:19 smithi134 bash[18722]: audit 2024-02-10T08:22:18.241518+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:22:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:19 smithi134 bash[18722]: audit 2024-02-10T08:22:18.242239+0000 mon.a (mon.0) 516 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:22:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:19 smithi134 bash[18722]: cluster 2024-02-10T08:22:18.913435+0000 mgr.y (mgr.14146) 277 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:19.594 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.587+0000 7ff70e7fc700 1 -- 172.21.15.134:0/3667369026 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff718101040 con 0x7ff704061280 2024-02-10T08:22:19.594 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 7 on host 'smithi134' 2024-02-10T08:22:19.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff704061280 msgr2=0x7ff704063740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff704061280 0x7ff704063740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7ff71810b290 tx=0x7ff714009e00).stop 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 msgr2=0x7ff718111330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718111330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff71000b890 tx=0x7ff710004d10).stop 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 shutdown_connections 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7ff704061280 0x7ff704063740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7180fb920 0x7ff718111330 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ff7180f9d30 0x7ff718109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 --2- 172.21.15.134:0/3667369026 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ff7180fafd0 0x7ff71810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 >> 172.21.15.134:0/3667369026 conn(0x7ff7180f5710 msgr2=0x7ff7180fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 shutdown_connections 2024-02-10T08:22:19.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:19.591+0000 7ff71f7e2700 1 -- 172.21.15.134:0/3667369026 wait complete. 2024-02-10T08:22:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[17385]: audit 2024-02-10T08:22:18.240603+0000 mon.a (mon.0) 513 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T08:22:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[17385]: cluster 2024-02-10T08:22:18.240718+0000 mon.a (mon.0) 514 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-10T08:22:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[17385]: audit 2024-02-10T08:22:18.240913+0000 mon.a (mon.0) 515 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[17385]: audit 2024-02-10T08:22:18.241518+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[17385]: audit 2024-02-10T08:22:18.242239+0000 mon.a (mon.0) 516 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[17385]: cluster 2024-02-10T08:22:18.913435+0000 mgr.y (mgr.14146) 277 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[24488]: audit 2024-02-10T08:22:18.240603+0000 mon.a (mon.0) 513 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[24488]: cluster 2024-02-10T08:22:18.240718+0000 mon.a (mon.0) 514 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[24488]: audit 2024-02-10T08:22:18.240913+0000 mon.a (mon.0) 515 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[24488]: audit 2024-02-10T08:22:18.241518+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[24488]: audit 2024-02-10T08:22:18.242239+0000 mon.a (mon.0) 516 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-02-10T08:22:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:19 smithi067 bash[24488]: cluster 2024-02-10T08:22:18.913435+0000 mgr.y (mgr.14146) 277 : cluster [DBG] pgmap v263: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:20.288 DEBUG:teuthology.orchestra.run.smithi134:osd.7> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.7.service 2024-02-10T08:22:20.290 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-10T08:22:20.291 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd stat -f json 2024-02-10T08:22:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:20 smithi134 bash[18722]: audit 2024-02-10T08:22:19.243605+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:22:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:20 smithi134 bash[18722]: cluster 2024-02-10T08:22:19.243752+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-10T08:22:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:20 smithi134 bash[18722]: audit 2024-02-10T08:22:19.244413+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:20 smithi134 bash[18722]: audit 2024-02-10T08:22:19.251945+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:20 smithi134 bash[18722]: audit 2024-02-10T08:22:19.592581+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:20 smithi134 bash[18722]: audit 2024-02-10T08:22:19.603026+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:20.550 INFO:journalctl@ceph.osd.7.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:22:20.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[17385]: audit 2024-02-10T08:22:19.243605+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:22:20.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[17385]: cluster 2024-02-10T08:22:19.243752+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-10T08:22:20.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[17385]: audit 2024-02-10T08:22:19.244413+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:20.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[17385]: audit 2024-02-10T08:22:19.251945+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:20.645 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[17385]: audit 2024-02-10T08:22:19.592581+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:20.646 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[17385]: audit 2024-02-10T08:22:19.603026+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:20.646 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[24488]: audit 2024-02-10T08:22:19.243605+0000 mon.a (mon.0) 517 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-02-10T08:22:20.646 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[24488]: cluster 2024-02-10T08:22:19.243752+0000 mon.a (mon.0) 518 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-10T08:22:20.646 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[24488]: audit 2024-02-10T08:22:19.244413+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:20.646 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[24488]: audit 2024-02-10T08:22:19.251945+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:20.647 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[24488]: audit 2024-02-10T08:22:19.592581+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:20.647 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:20 smithi067 bash[24488]: audit 2024-02-10T08:22:19.603026+0000 mon.a (mon.0) 522 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: cluster 2024-02-10T08:22:18.450910+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:22:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: cluster 2024-02-10T08:22:18.451042+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:22:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: audit 2024-02-10T08:22:20.249710+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: cluster 2024-02-10T08:22:20.255742+0000 mon.a (mon.0) 524 : cluster [INF] osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] boot 2024-02-10T08:22:21.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: cluster 2024-02-10T08:22:20.255826+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-02-10T08:22:21.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: audit 2024-02-10T08:22:20.256654+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:21.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:21 smithi134 bash[18722]: cluster 2024-02-10T08:22:20.913950+0000 mgr.y (mgr.14146) 278 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:21.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: cluster 2024-02-10T08:22:18.450910+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:22:21.550 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: cluster 2024-02-10T08:22:18.451042+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: audit 2024-02-10T08:22:20.249710+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: cluster 2024-02-10T08:22:20.255742+0000 mon.a (mon.0) 524 : cluster [INF] osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] boot 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: cluster 2024-02-10T08:22:20.255826+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: audit 2024-02-10T08:22:20.256654+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[17385]: cluster 2024-02-10T08:22:20.913950+0000 mgr.y (mgr.14146) 278 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: cluster 2024-02-10T08:22:18.450910+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: cluster 2024-02-10T08:22:18.451042+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: audit 2024-02-10T08:22:20.249710+0000 mon.a (mon.0) 523 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: cluster 2024-02-10T08:22:20.255742+0000 mon.a (mon.0) 524 : cluster [INF] osd.7 [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] boot 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: cluster 2024-02-10T08:22:20.255826+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-02-10T08:22:21.551 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: audit 2024-02-10T08:22:20.256654+0000 mon.a (mon.0) 526 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:22:21.552 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:21 smithi067 bash[24488]: cluster 2024-02-10T08:22:20.913950+0000 mgr.y (mgr.14146) 278 : cluster [DBG] pgmap v266: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-10T08:22:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:22 smithi134 bash[18722]: cluster 2024-02-10T08:22:21.267936+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-10T08:22:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:22 smithi067 bash[17385]: cluster 2024-02-10T08:22:21.267936+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-10T08:22:22.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:22 smithi067 bash[24488]: cluster 2024-02-10T08:22:21.267936+0000 mon.a (mon.0) 527 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-10T08:22:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 -- 172.21.15.67:0/726092222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f6790 msgr2=0x7fab980f8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 --2- 172.21.15.67:0/726092222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f6790 0x7fab980f8f10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fab94004870 tx=0x7fab94009cb0).stop 2024-02-10T08:22:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 -- 172.21.15.67:0/726092222 shutdown_connections 2024-02-10T08:22:23.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 --2- 172.21.15.67:0/726092222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab980f9530 0x7fab980f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 --2- 172.21.15.67:0/726092222 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f5df0 0x7fab980f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 --2- 172.21.15.67:0/726092222 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f6790 0x7fab980f8f10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 -- 172.21.15.67:0/726092222 >> 172.21.15.67:0/726092222 conn(0x7fab980f19d0 msgr2=0x7fab980f3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 -- 172.21.15.67:0/726092222 shutdown_connections 2024-02-10T08:22:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.436+0000 7faba018f700 1 -- 172.21.15.67:0/726092222 wait complete. 2024-02-10T08:22:23.443 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 Processor -- start 2024-02-10T08:22:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 -- start start 2024-02-10T08:22:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f5df0 0x7fab980f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f6790 0x7fab980f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:23.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab980f9530 0x7fab980f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab980f7930 con 0x7fab980f5df0 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab980f7aa0 con 0x7fab980f9530 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab980f7c60 con 0x7fab980f6790 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9df2b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f5df0 0x7fab980f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f6790 0x7fab980f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9df2b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f5df0 0x7fab980f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:44170/0 (socket says 172.21.15.67:44170) 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9df2b700 1 -- 172.21.15.67:0/1651722199 learned_addr learned my addr 172.21.15.67:0/1651722199 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:22:23.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9e72c700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab980f9530 0x7fab980f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 -- 172.21.15.67:0/1651722199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab980f9530 msgr2=0x7fab980f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab980f9530 0x7fab980f7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 -- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f5df0 msgr2=0x7fab980f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f5df0 0x7fab980f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 -- 172.21.15.67:0/1651722199 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94005040 con 0x7fab980f6790 2024-02-10T08:22:23.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab9d72a700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f6790 0x7fab980f6e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fab94004710 tx=0x7fab94009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400e030 con 0x7fab980f6790 2024-02-10T08:22:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fab940096d0 con 0x7fab980f6790 2024-02-10T08:22:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab981121e0 con 0x7fab980f6790 2024-02-10T08:22:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94018070 con 0x7fab980f6790 2024-02-10T08:22:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.440+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab98112660 con 0x7fab980f6790 2024-02-10T08:22:23.447 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.444+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab98060070 con 0x7fab980f6790 2024-02-10T08:22:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.444+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fab94009240 con 0x7fab980f6790 2024-02-10T08:22:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.444+0000 7fab8affd700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fab84061250 0x7fab84063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.444+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5260+0+0 (secure 0 0 0) 0x7fab9407d520 con 0x7fab980f6790 2024-02-10T08:22:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.444+0000 7fab9df2b700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fab84061250 0x7fab84063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.444+0000 7fab9df2b700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fab84061250 0x7fab84063710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fab8c0041d0 tx=0x7fab8c00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:23.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.452+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fab9404db90 con 0x7fab980f6790 2024-02-10T08:22:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:23 smithi067 bash[24488]: cluster 2024-02-10T08:22:22.272617+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-10T08:22:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:23 smithi067 bash[24488]: cluster 2024-02-10T08:22:22.914522+0000 mgr.y (mgr.14146) 279 : cluster [DBG] pgmap v269: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:23 smithi067 bash[17385]: cluster 2024-02-10T08:22:22.272617+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-10T08:22:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:23 smithi067 bash[17385]: cluster 2024-02-10T08:22:22.914522+0000 mgr.y (mgr.14146) 279 : cluster [DBG] pgmap v269: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:23.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.636+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fab980fee70 con 0x7fab980f6790 2024-02-10T08:22:23.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.636+0000 7fab8affd700 1 -- 172.21.15.67:0/1651722199 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v50) v1 ==== 74+0+130 (secure 0 0 0) 0x7fab940511b0 con 0x7fab980f6790 2024-02-10T08:22:23.641 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:22:23.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fab84061250 msgr2=0x7fab84063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:23.645 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fab84061250 0x7fab84063710 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fab8c0041d0 tx=0x7fab8c00b040).stop 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f6790 msgr2=0x7fab980f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f6790 0x7fab980f6e20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fab94004710 tx=0x7fab94009cb0).stop 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 shutdown_connections 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fab84061250 0x7fab84063710 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fab980f5df0 0x7fab980f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab980f9530 0x7fab980f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 --2- 172.21.15.67:0/1651722199 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fab980f6790 0x7fab980f6e20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 >> 172.21.15.67:0/1651722199 conn(0x7fab980f19d0 msgr2=0x7fab980fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 shutdown_connections 2024-02-10T08:22:23.646 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:22:23.640+0000 7faba018f700 1 -- 172.21.15.67:0/1651722199 wait complete. 2024-02-10T08:22:23.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:23 smithi134 bash[18722]: cluster 2024-02-10T08:22:22.272617+0000 mon.a (mon.0) 528 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-10T08:22:23.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:23 smithi134 bash[18722]: cluster 2024-02-10T08:22:22.914522+0000 mgr.y (mgr.14146) 279 : cluster [DBG] pgmap v269: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:24.361 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1707553340,"num_in_osds":8,"osd_in_since":1707553314,"num_remapped_pgs":0} 2024-02-10T08:22:24.362 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi067 2024-02-10T08:22:24.362 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi067=foo.a' 2024-02-10T08:22:24.554 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:24 smithi134 bash[18722]: audit 2024-02-10T08:22:23.641157+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.67:0/1651722199' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T08:22:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:24 smithi067 bash[17385]: audit 2024-02-10T08:22:23.641157+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.67:0/1651722199' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T08:22:24.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:24 smithi067 bash[24488]: audit 2024-02-10T08:22:23.641157+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.67:0/1651722199' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T08:22:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:25 smithi067 bash[17385]: cluster 2024-02-10T08:22:24.915176+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:25.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:25 smithi067 bash[24488]: cluster 2024-02-10T08:22:24.915176+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:25.765 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:25 smithi134 bash[18722]: cluster 2024-02-10T08:22:24.915176+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v270: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:28.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:28 smithi067 bash[17385]: cluster 2024-02-10T08:22:26.915839+0000 mgr.y (mgr.14146) 281 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:28.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:28 smithi067 bash[24488]: cluster 2024-02-10T08:22:26.915839+0000 mgr.y (mgr.14146) 281 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- 172.21.15.134:0/3139288909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 msgr2=0x7fc81c0f8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- 172.21.15.134:0/3139288909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0f8130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc80c0048a0 tx=0x7fc80c009cb0).stop 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- 172.21.15.134:0/3139288909 shutdown_connections 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- 172.21.15.134:0/3139288909 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc81c0f6a30 0x7fc81c0f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- 172.21.15.134:0/3139288909 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc81c0f8620 0x7fc81c0fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- 172.21.15.134:0/3139288909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0f8130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- 172.21.15.134:0/3139288909 >> 172.21.15.134:0/3139288909 conn(0x7fc81c0f2230 msgr2=0x7fc81c0f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- 172.21.15.134:0/3139288909 shutdown_connections 2024-02-10T08:22:28.477 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- 172.21.15.134:0/3139288909 wait complete. 2024-02-10T08:22:28.478 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 Processor -- start 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- start start 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc81c0f6a30 0x7fc81c101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc81c0f8620 0x7fc81c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c100370 con 0x7fc81c0f6a30 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1004e0 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc8215a1700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1006a0 con 0x7fc81c0f8620 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39142/0 (socket says 172.21.15.134:39142) 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 -- 172.21.15.134:0/2655537063 learned_addr learned my addr 172.21.15.134:0/2655537063 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81b7fe700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc81c0f8620 0x7fc81c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:28.479 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81affd700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc81c0f6a30 0x7fc81c101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:28.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 -- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc81c0f8620 msgr2=0x7fc81c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:28.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc81c0f8620 0x7fc81c0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 -- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc81c0f6a30 msgr2=0x7fc81c101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:28.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc81c0f6a30 0x7fc81c101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.471+0000 7fc81a7fc700 1 -- 172.21.15.134:0/2655537063 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c005040 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.480 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc81a7fc700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0ff860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc80c005010 tx=0x7fc80c009cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:28.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c00e030 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c112040 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c112520 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fc80c013e30 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c018070 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.481 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808004f50 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fc80c009240 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc803fff700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fc804061250 0x7fc804063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:28.482 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5260+0+0 (secure 0 0 0) 0x7fc80c07d760 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.475+0000 7fc81affd700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fc804061250 0x7fc804063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:28.483 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.479+0000 7fc81affd700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fc804061250 0x7fc804063710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fc81c0f7b30 tx=0x7fc81000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:28.490 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.483+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc80c04dd30 con 0x7fc81c0f7cd0 2024-02-10T08:22:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:28 smithi134 bash[18722]: cluster 2024-02-10T08:22:26.915839+0000 mgr.y (mgr.14146) 281 : cluster [DBG] pgmap v271: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:28.685 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.679+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc808000bf0 con 0x7fc804061250 2024-02-10T08:22:28.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc803fff700 1 -- 172.21.15.134:0/2655537063 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fc808000bf0 con 0x7fc804061250 2024-02-10T08:22:28.699 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled rgw.foo.a update... 2024-02-10T08:22:28.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fc804061250 msgr2=0x7fc804063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:28.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fc804061250 0x7fc804063710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fc81c0f7b30 tx=0x7fc81000b040).stop 2024-02-10T08:22:28.702 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 msgr2=0x7fc81c0ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0ff860 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fc80c005010 tx=0x7fc80c009cb0).stop 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 shutdown_connections 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fc804061250 0x7fc804063710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc81c0f6a30 0x7fc81c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc81c0f8620 0x7fc81c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 --2- 172.21.15.134:0/2655537063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc81c0f7cd0 0x7fc81c0ff860 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 >> 172.21.15.134:0/2655537063 conn(0x7fc81c0f2230 msgr2=0x7fc81c0f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 shutdown_connections 2024-02-10T08:22:28.703 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:28.695+0000 7fc8215a1700 1 -- 172.21.15.134:0/2655537063 wait complete. 2024-02-10T08:22:29.606 DEBUG:teuthology.orchestra.run.smithi067:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@rgw.foo.a.service 2024-02-10T08:22:29.609 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi134 2024-02-10T08:22:29.610 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool create iscsi 3 3 replicated 2024-02-10T08:22:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:29 smithi134 bash[18722]: audit 2024-02-10T08:22:28.687841+0000 mgr.y (mgr.14146) 282 : audit [DBG] from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:29 smithi134 bash[18722]: cephadm 2024-02-10T08:22:28.691719+0000 mgr.y (mgr.14146) 283 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-10T08:22:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:29 smithi134 bash[18722]: audit 2024-02-10T08:22:28.698057+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:29 smithi134 bash[18722]: cluster 2024-02-10T08:22:28.916426+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v272: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:29.854 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:22:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[17385]: audit 2024-02-10T08:22:28.687841+0000 mgr.y (mgr.14146) 282 : audit [DBG] from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[17385]: cephadm 2024-02-10T08:22:28.691719+0000 mgr.y (mgr.14146) 283 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-10T08:22:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[17385]: audit 2024-02-10T08:22:28.698057+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[17385]: cluster 2024-02-10T08:22:28.916426+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v272: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[24488]: audit 2024-02-10T08:22:28.687841+0000 mgr.y (mgr.14146) 282 : audit [DBG] from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[24488]: cephadm 2024-02-10T08:22:28.691719+0000 mgr.y (mgr.14146) 283 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-10T08:22:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[24488]: audit 2024-02-10T08:22:28.698057+0000 mon.a (mon.0) 529 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:29 smithi067 bash[24488]: cluster 2024-02-10T08:22:28.916426+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v272: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:31.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: cephadm 2024-02-10T08:22:30.505626+0000 mgr.y (mgr.14146) 285 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.515429+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.517443+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.519024+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.520377+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.521655+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: cephadm 2024-02-10T08:22:30.522627+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 4316M 2024-02-10T08:22:31.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.531091+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.532656+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.533946+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.543272+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: cephadm 2024-02-10T08:22:30.547128+0000 mgr.y (mgr.14146) 287 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.554869+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.555882+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.564241+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.573451+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: audit 2024-02-10T08:22:30.574990+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: cephadm 2024-02-10T08:22:30.576073+0000 mgr.y (mgr.14146) 288 : cephadm [INF] Deploying daemon rgw.foo.a on smithi067 2024-02-10T08:22:31.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:31 smithi134 bash[18722]: cluster 2024-02-10T08:22:30.917059+0000 mgr.y (mgr.14146) 289 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:31.794 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: cephadm 2024-02-10T08:22:30.505626+0000 mgr.y (mgr.14146) 285 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.515429+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.517443+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.519024+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.520377+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.521655+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: cephadm 2024-02-10T08:22:30.522627+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 4316M 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.531091+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.532656+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.533946+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.543272+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: cephadm 2024-02-10T08:22:30.547128+0000 mgr.y (mgr.14146) 287 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.554869+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.555882+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.564241+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.573451+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: audit 2024-02-10T08:22:30.574990+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: cephadm 2024-02-10T08:22:30.576073+0000 mgr.y (mgr.14146) 288 : cephadm [INF] Deploying daemon rgw.foo.a on smithi067 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[24488]: cluster 2024-02-10T08:22:30.917059+0000 mgr.y (mgr.14146) 289 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: cephadm 2024-02-10T08:22:30.505626+0000 mgr.y (mgr.14146) 285 : cephadm [INF] Detected new or changed devices on smithi134 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.515429+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.517443+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.519024+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.520377+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.796 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.521655+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: cephadm 2024-02-10T08:22:30.522627+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 4316M 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.531091+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.532656+0000 mon.a (mon.0) 536 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.533946+0000 mon.a (mon.0) 537 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.543272+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: cephadm 2024-02-10T08:22:30.547128+0000 mgr.y (mgr.14146) 287 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.554869+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.555882+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.564241+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.573451+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: audit 2024-02-10T08:22:30.574990+0000 mon.a (mon.0) 543 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: cephadm 2024-02-10T08:22:30.576073+0000 mgr.y (mgr.14146) 288 : cephadm [INF] Deploying daemon rgw.foo.a on smithi067 2024-02-10T08:22:31.797 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:31 smithi067 bash[17385]: cluster 2024-02-10T08:22:30.917059+0000 mgr.y (mgr.14146) 289 : cluster [DBG] pgmap v273: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:32.595 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 -- 172.21.15.134:0/3531448206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c0061ca0 msgr2=0x7fa1c00fa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:32.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/3531448206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c0061ca0 0x7fa1c00fa7e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fa1b0004870 tx=0x7fa1b0009cb0).stop 2024-02-10T08:22:32.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 -- 172.21.15.134:0/3531448206 shutdown_connections 2024-02-10T08:22:32.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/3531448206 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1c00fd700 0x7fa1c00ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:32.596 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/3531448206 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c0061ca0 0x7fa1c00fa7e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:32.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/3531448206 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00facd0 0x7fa1c00fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:32.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 -- 172.21.15.134:0/3531448206 >> 172.21.15.134:0/3531448206 conn(0x7fa1c00f2220 msgr2=0x7fa1c00f4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:32.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 -- 172.21.15.134:0/3531448206 shutdown_connections 2024-02-10T08:22:32.597 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.587+0000 7fa1c5c26700 1 -- 172.21.15.134:0/3531448206 wait complete. 2024-02-10T08:22:32.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 Processor -- start 2024-02-10T08:22:32.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 -- start start 2024-02-10T08:22:32.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1c0061ca0 0x7fa1c0107f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:32.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 0x7fa1c01083f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:32.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00fd700 0x7fa1c010f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:32.600 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c010f9c0 con 0x7fa1c00fd700 2024-02-10T08:22:32.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c010fb00 con 0x7fa1c00facd0 2024-02-10T08:22:32.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c010fc40 con 0x7fa1c0061ca0 2024-02-10T08:22:32.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 0x7fa1c01083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:32.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 0x7fa1c01083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39906/0 (socket says 172.21.15.134:39906) 2024-02-10T08:22:32.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 -- 172.21.15.134:0/436568890 learned_addr learned my addr 172.21.15.134:0/436568890 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:32.601 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1bf7fe700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1c0061ca0 0x7fa1c0107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1bffff700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00fd700 0x7fa1c010f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 -- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1c0061ca0 msgr2=0x7fa1c0107f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1c0061ca0 0x7fa1c0107f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 -- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00fd700 msgr2=0x7fa1c010f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00fd700 0x7fa1c010f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 -- 172.21.15.134:0/436568890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0005040 con 0x7fa1c00facd0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1bffff700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00fd700 0x7fa1c010f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1b7fff700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 0x7fa1c01083f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa1a80041d0 tx=0x7fa1a8009f60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8007b40 con 0x7fa1c00facd0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c010ff20 con 0x7fa1c00facd0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa1a800f040 con 0x7fa1c00facd0 2024-02-10T08:22:32.602 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8014690 con 0x7fa1c00facd0 2024-02-10T08:22:32.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.591+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c0110450 con 0x7fa1c00facd0 2024-02-10T08:22:32.603 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.595+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c0102750 con 0x7fa1c00facd0 2024-02-10T08:22:32.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.595+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7fa1a8009110 con 0x7fa1c00facd0 2024-02-10T08:22:32.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.595+0000 7fa1bd7fa700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa1a0061250 0x7fa1a0063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:32.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.595+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5260+0+0 (secure 0 0 0) 0x7fa1a807c580 con 0x7fa1c00facd0 2024-02-10T08:22:32.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.595+0000 7fa1bf7fe700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa1a0061250 0x7fa1a0063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:32.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.599+0000 7fa1bf7fe700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa1a0061250 0x7fa1a0063710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa1b0009f20 tx=0x7fa1b0009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:32.608 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.603+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa1a804cb50 con 0x7fa1c00facd0 2024-02-10T08:22:32.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:32.787+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fa1c01028e0 con 0x7fa1c00facd0 2024-02-10T08:22:33.201 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 10 08:22:33 smithi067 systemd[1]: Started Ceph rgw.foo.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:22:33.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.335+0000 7fa1bd7fa700 1 -- 172.21.15.134:0/436568890 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'iscsi' created v51) v1 ==== 154+0+0 (secure 0 0 0) 0x7fa1c01028e0 con 0x7fa1c00facd0 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa1a0061250 msgr2=0x7fa1a0063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa1a0061250 0x7fa1a0063710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7fa1b0009f20 tx=0x7fa1b0009cb0).stop 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 msgr2=0x7fa1c01083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 0x7fa1c01083f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa1a80041d0 tx=0x7fa1a8009f60).stop 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 shutdown_connections 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7fa1a0061250 0x7fa1a0063710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1c00fd700 0x7fa1c010f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1c0061ca0 0x7fa1c0107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 --2- 172.21.15.134:0/436568890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1c00facd0 0x7fa1c01083f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 >> 172.21.15.134:0/436568890 conn(0x7fa1c00f2220 msgr2=0x7fa1c00fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:33.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 shutdown_connections 2024-02-10T08:22:33.346 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:33.339+0000 7fa1c5c26700 1 -- 172.21.15.134:0/436568890 wait complete. 2024-02-10T08:22:33.347 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'iscsi' created 2024-02-10T08:22:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[17385]: audit 2024-02-10T08:22:32.791906+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.134:0/436568890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T08:22:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[17385]: audit 2024-02-10T08:22:32.793161+0000 mon.a (mon.0) 544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T08:22:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[17385]: cluster 2024-02-10T08:22:32.917724+0000 mgr.y (mgr.14146) 290 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[17385]: audit 2024-02-10T08:22:33.289526+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[17385]: audit 2024-02-10T08:22:33.291076+0000 mon.a (mon.0) 546 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[24488]: audit 2024-02-10T08:22:32.791906+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.134:0/436568890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T08:22:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[24488]: audit 2024-02-10T08:22:32.793161+0000 mon.a (mon.0) 544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T08:22:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[24488]: cluster 2024-02-10T08:22:32.917724+0000 mgr.y (mgr.14146) 290 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[24488]: audit 2024-02-10T08:22:33.289526+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:33 smithi067 bash[24488]: audit 2024-02-10T08:22:33.291076+0000 mon.a (mon.0) 546 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:33 smithi134 bash[18722]: audit 2024-02-10T08:22:32.791906+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.134:0/436568890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T08:22:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:33 smithi134 bash[18722]: audit 2024-02-10T08:22:32.793161+0000 mon.a (mon.0) 544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-10T08:22:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:33 smithi134 bash[18722]: cluster 2024-02-10T08:22:32.917724+0000 mgr.y (mgr.14146) 290 : cluster [DBG] pgmap v274: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:33 smithi134 bash[18722]: audit 2024-02-10T08:22:33.289526+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:33 smithi134 bash[18722]: audit 2024-02-10T08:22:33.291076+0000 mon.a (mon.0) 546 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:34.039 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool application enable iscsi rbd 2024-02-10T08:22:35.048 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:34 smithi134 bash[18722]: audit 2024-02-10T08:22:33.340129+0000 mon.a (mon.0) 547 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T08:22:35.049 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:34 smithi134 bash[18722]: cluster 2024-02-10T08:22:33.340299+0000 mon.a (mon.0) 548 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-10T08:22:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:34 smithi067 bash[17385]: audit 2024-02-10T08:22:33.340129+0000 mon.a (mon.0) 547 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T08:22:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:34 smithi067 bash[17385]: cluster 2024-02-10T08:22:33.340299+0000 mon.a (mon.0) 548 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-10T08:22:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:34 smithi067 bash[24488]: audit 2024-02-10T08:22:33.340129+0000 mon.a (mon.0) 547 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-10T08:22:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:34 smithi067 bash[24488]: cluster 2024-02-10T08:22:33.340299+0000 mon.a (mon.0) 548 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-10T08:22:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[17385]: cluster 2024-02-10T08:22:34.790103+0000 mon.a (mon.0) 549 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[17385]: audit 2024-02-10T08:22:34.792565+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[17385]: audit 2024-02-10T08:22:34.798446+0000 mon.a (mon.0) 550 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[17385]: cluster 2024-02-10T08:22:34.918343+0000 mgr.y (mgr.14146) 291 : cluster [DBG] pgmap v277: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[24488]: cluster 2024-02-10T08:22:34.790103+0000 mon.a (mon.0) 549 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[24488]: audit 2024-02-10T08:22:34.792565+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[24488]: audit 2024-02-10T08:22:34.798446+0000 mon.a (mon.0) 550 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T08:22:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:35 smithi067 bash[24488]: cluster 2024-02-10T08:22:34.918343+0000 mgr.y (mgr.14146) 291 : cluster [DBG] pgmap v277: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:35 smithi134 bash[18722]: cluster 2024-02-10T08:22:34.790103+0000 mon.a (mon.0) 549 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-10T08:22:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:35 smithi134 bash[18722]: audit 2024-02-10T08:22:34.792565+0000 mon.c (mon.2) 13 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T08:22:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:35 smithi134 bash[18722]: audit 2024-02-10T08:22:34.798446+0000 mon.a (mon.0) 550 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T08:22:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:35 smithi134 bash[18722]: cluster 2024-02-10T08:22:34.918343+0000 mgr.y (mgr.14146) 291 : cluster [DBG] pgmap v277: 36 pgs: 35 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 -- 172.21.15.134:0/3461210309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 msgr2=0x7f38480f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 --2- 172.21.15.134:0/3461210309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f38480f7240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f3830004870 tx=0x7f3830009cb0).stop 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 -- 172.21.15.134:0/3461210309 shutdown_connections 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 --2- 172.21.15.134:0/3461210309 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38480f7730 0x7f38480ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 --2- 172.21.15.134:0/3461210309 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f38480f80b0 0x7f38480f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 --2- 172.21.15.134:0/3461210309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f38480f7240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 -- 172.21.15.134:0/3461210309 >> 172.21.15.134:0/3461210309 conn(0x7f38480f19d0 msgr2=0x7f38480f3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 -- 172.21.15.134:0/3461210309 shutdown_connections 2024-02-10T08:22:36.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.859+0000 7f384db5d700 1 -- 172.21.15.134:0/3461210309 wait complete. 2024-02-10T08:22:36.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 Processor -- start 2024-02-10T08:22:36.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- start start 2024-02-10T08:22:36.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f3848106df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:36.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38480f7730 0x7f38481072e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:36.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f38480f80b0 0x7f384810e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:36.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384810e8b0 con 0x7f38480f7730 2024-02-10T08:22:36.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384810e9f0 con 0x7f38480f6dc0 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38480f5cb0 con 0x7f38480f80b0 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f3848106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f3848106df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39924/0 (socket says 172.21.15.134:39924) 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 -- 172.21.15.134:0/3367277946 learned_addr learned my addr 172.21.15.134:0/3367277946 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f383effd700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38480f7730 0x7f38481072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f3847fff700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f38480f80b0 0x7f384810e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 -- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f38480f80b0 msgr2=0x7f384810e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f38480f80b0 0x7f384810e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 -- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38480f7730 msgr2=0x7f38481072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38480f7730 0x7f38481072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 -- 172.21.15.134:0/3367277946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830005040 con 0x7f38480f6dc0 2024-02-10T08:22:36.869 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38477fe700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f3848106df0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f383800eca0 tx=0x7f383800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:36.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838009950 con 0x7f38480f6dc0 2024-02-10T08:22:36.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f383800f040 con 0x7f38480f6dc0 2024-02-10T08:22:36.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38480f5f30 con 0x7f38480f6dc0 2024-02-10T08:22:36.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838007d90 con 0x7f38480f6dc0 2024-02-10T08:22:36.870 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38480f64a0 con 0x7f38480f6dc0 2024-02-10T08:22:36.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f383800c9a0 con 0x7f38480f6dc0 2024-02-10T08:22:36.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.863+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3848060070 con 0x7f38480f6dc0 2024-02-10T08:22:36.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.867+0000 7f38457fa700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3834061250 0x7f3834063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:36.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.867+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6317+0+0 (secure 0 0 0) 0x7f383807c920 con 0x7f38480f6dc0 2024-02-10T08:22:36.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.867+0000 7f383effd700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3834061250 0x7f3834063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:36.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.867+0000 7f383effd700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3834061250 0x7f3834063710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3830009920 tx=0x7f383000b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:36.877 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:36.871+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3838049f60 con 0x7f38480f6dc0 2024-02-10T08:22:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:36 smithi134 bash[18722]: audit 2024-02-10T08:22:35.794046+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T08:22:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:36 smithi134 bash[18722]: cluster 2024-02-10T08:22:35.794324+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-10T08:22:37.060 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.055+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"} v 0) v1 -- 0x7f3848101660 con 0x7f38480f6dc0 2024-02-10T08:22:37.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:36 smithi067 bash[24488]: audit 2024-02-10T08:22:35.794046+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T08:22:37.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:36 smithi067 bash[24488]: cluster 2024-02-10T08:22:35.794324+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-10T08:22:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:36 smithi067 bash[17385]: audit 2024-02-10T08:22:35.794046+0000 mon.a (mon.0) 551 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T08:22:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:36 smithi067 bash[17385]: cluster 2024-02-10T08:22:35.794324+0000 mon.a (mon.0) 552 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-10T08:22:37.812 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.807+0000 7f38457fa700 1 -- 172.21.15.134:0/3367277946 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]=0 enabled application 'rbd' on pool 'iscsi' v55) v1 ==== 147+0+0 (secure 0 0 0) 0x7f383804d7c0 con 0x7f38480f6dc0 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3834061250 msgr2=0x7f3834063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3834061250 0x7f3834063710 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f3830009920 tx=0x7f383000b040).stop 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 msgr2=0x7f3848106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f3848106df0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f383800eca0 tx=0x7f383800b040).stop 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 shutdown_connections 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3834061250 0x7f3834063710 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f38480f80b0 0x7f384810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:37.817 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38480f6dc0 0x7f3848106df0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:37.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 --2- 172.21.15.134:0/3367277946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f38480f7730 0x7f38481072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:37.818 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.811+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 >> 172.21.15.134:0/3367277946 conn(0x7f38480f19d0 msgr2=0x7f38480fb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:37.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.815+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 shutdown_connections 2024-02-10T08:22:37.819 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:37.815+0000 7f384db5d700 1 -- 172.21.15.134:0/3367277946 wait complete. 2024-02-10T08:22:37.820 INFO:teuthology.orchestra.run.smithi134.stderr:enabled application 'rbd' on pool 'iscsi' 2024-02-10T08:22:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: cluster 2024-02-10T08:22:36.808414+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-10T08:22:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:36.811352+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T08:22:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:36.818440+0000 mon.a (mon.0) 554 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: cluster 2024-02-10T08:22:36.919032+0000 mgr.y (mgr.14146) 292 : cluster [DBG] pgmap v280: 68 pgs: 6 creating+peering, 12 active+clean, 50 unknown; 450 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:36.944835+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:37.060107+0000 mon.b (mon.1) 20 : audit [INF] from='client.? 172.21.15.134:0/3367277946' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:37.061203+0000 mon.a (mon.0) 556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:37.810274+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: audit 2024-02-10T08:22:37.811068+0000 mon.a (mon.0) 558 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[17385]: cluster 2024-02-10T08:22:37.811209+0000 mon.a (mon.0) 559 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: cluster 2024-02-10T08:22:36.808414+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:36.811352+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:36.818440+0000 mon.a (mon.0) 554 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: cluster 2024-02-10T08:22:36.919032+0000 mgr.y (mgr.14146) 292 : cluster [DBG] pgmap v280: 68 pgs: 6 creating+peering, 12 active+clean, 50 unknown; 450 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:36.944835+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:37.060107+0000 mon.b (mon.1) 20 : audit [INF] from='client.? 172.21.15.134:0/3367277946' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-10T08:22:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:37.061203+0000 mon.a (mon.0) 556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-10T08:22:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:37.810274+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T08:22:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: audit 2024-02-10T08:22:37.811068+0000 mon.a (mon.0) 558 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-10T08:22:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:37 smithi067 bash[24488]: cluster 2024-02-10T08:22:37.811209+0000 mon.a (mon.0) 559 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-10T08:22:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: cluster 2024-02-10T08:22:36.808414+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-10T08:22:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:36.811352+0000 mon.c (mon.2) 14 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T08:22:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:36.818440+0000 mon.a (mon.0) 554 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T08:22:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: cluster 2024-02-10T08:22:36.919032+0000 mgr.y (mgr.14146) 292 : cluster [DBG] pgmap v280: 68 pgs: 6 creating+peering, 12 active+clean, 50 unknown; 450 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:22:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:36.944835+0000 mon.a (mon.0) 555 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:37.060107+0000 mon.b (mon.1) 20 : audit [INF] from='client.? 172.21.15.134:0/3367277946' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-10T08:22:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:37.061203+0000 mon.a (mon.0) 556 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-10T08:22:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:37.810274+0000 mon.a (mon.0) 557 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T08:22:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: audit 2024-02-10T08:22:37.811068+0000 mon.a (mon.0) 558 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-10T08:22:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:37 smithi134 bash[18722]: cluster 2024-02-10T08:22:37.811209+0000 mon.a (mon.0) 559 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-10T08:22:38.575 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply iscsi iscsi user password --placement '1;smithi134=iscsi.a' 2024-02-10T08:22:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: audit 2024-02-10T08:22:38.641154+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: cluster 2024-02-10T08:22:38.830214+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-10T08:22:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: audit 2024-02-10T08:22:38.833963+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T08:22:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: audit 2024-02-10T08:22:38.838054+0000 mon.a (mon.0) 562 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T08:22:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: audit 2024-02-10T08:22:38.906827+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: audit 2024-02-10T08:22:38.908438+0000 mon.a (mon.0) 564 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: cephadm 2024-02-10T08:22:38.918133+0000 mgr.y (mgr.14146) 293 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-10T08:22:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:39 smithi134 bash[18722]: cluster 2024-02-10T08:22:38.926981+0000 mgr.y (mgr.14146) 294 : cluster [DBG] pgmap v283: 100 pgs: 13 creating+peering, 24 active+clean, 63 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 964 B/s rd, 482 B/s wr, 1 op/s 2024-02-10T08:22:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: audit 2024-02-10T08:22:38.641154+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: cluster 2024-02-10T08:22:38.830214+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-10T08:22:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: audit 2024-02-10T08:22:38.833963+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T08:22:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: audit 2024-02-10T08:22:38.838054+0000 mon.a (mon.0) 562 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: audit 2024-02-10T08:22:38.906827+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: audit 2024-02-10T08:22:38.908438+0000 mon.a (mon.0) 564 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: cephadm 2024-02-10T08:22:38.918133+0000 mgr.y (mgr.14146) 293 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[17385]: cluster 2024-02-10T08:22:38.926981+0000 mgr.y (mgr.14146) 294 : cluster [DBG] pgmap v283: 100 pgs: 13 creating+peering, 24 active+clean, 63 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 964 B/s rd, 482 B/s wr, 1 op/s 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: audit 2024-02-10T08:22:38.641154+0000 mon.a (mon.0) 560 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: cluster 2024-02-10T08:22:38.830214+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: audit 2024-02-10T08:22:38.833963+0000 mon.c (mon.2) 15 : audit [INF] from='client.? 172.21.15.67:0/2701903425' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: audit 2024-02-10T08:22:38.838054+0000 mon.a (mon.0) 562 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: audit 2024-02-10T08:22:38.906827+0000 mon.a (mon.0) 563 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: audit 2024-02-10T08:22:38.908438+0000 mon.a (mon.0) 564 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: cephadm 2024-02-10T08:22:38.918133+0000 mgr.y (mgr.14146) 293 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-10T08:22:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:39 smithi067 bash[24488]: cluster 2024-02-10T08:22:38.926981+0000 mgr.y (mgr.14146) 294 : cluster [DBG] pgmap v283: 100 pgs: 13 creating+peering, 24 active+clean, 63 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 964 B/s rd, 482 B/s wr, 1 op/s 2024-02-10T08:22:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:40 smithi134 bash[18722]: audit 2024-02-10T08:22:39.835039+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T08:22:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:40 smithi134 bash[18722]: cluster 2024-02-10T08:22:39.835249+0000 mon.a (mon.0) 566 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- 172.21.15.134:0/1355059123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f5e70 msgr2=0x7f52900f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- 172.21.15.134:0/1355059123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f5e70 0x7f52900f6290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5280004870 tx=0x7f5280009cb0).stop 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- 172.21.15.134:0/1355059123 shutdown_connections 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- 172.21.15.134:0/1355059123 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52900f9720 0x7f52900f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- 172.21.15.134:0/1355059123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f5e70 0x7f52900f6290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- 172.21.15.134:0/1355059123 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52900f6810 0x7f52900f9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- 172.21.15.134:0/1355059123 >> 172.21.15.134:0/1355059123 conn(0x7f52900f1970 msgr2=0x7f52900f3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- 172.21.15.134:0/1355059123 shutdown_connections 2024-02-10T08:22:41.289 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- 172.21.15.134:0/1355059123 wait complete. 2024-02-10T08:22:41.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 Processor -- start 2024-02-10T08:22:41.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- start start 2024-02-10T08:22:41.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52900f5e70 0x7f52900f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:41.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52900f6810 0x7f52900f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:41.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 0x7f52900f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:41.290 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900f79f0 con 0x7f52900f6810 2024-02-10T08:22:41.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52900f7b60 con 0x7f52900f9720 2024-02-10T08:22:41.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5296eec700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900f7d20 con 0x7f52900f5e70 2024-02-10T08:22:41.291 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5295489700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 0x7f52900f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5295489700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 0x7f52900f73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:46314/0 (socket says 172.21.15.134:46314) 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5295489700 1 -- 172.21.15.134:0/3541758129 learned_addr learned my addr 172.21.15.134:0/3541758129 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5287fff700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52900f6810 0x7f52900f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.283+0000 7f5294c88700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52900f5e70 0x7f52900f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5295489700 1 -- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52900f5e70 msgr2=0x7f52900f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5295489700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52900f5e70 0x7f52900f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5295489700 1 -- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52900f6810 msgr2=0x7f52900f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5295489700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52900f6810 0x7f52900f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.292 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5295489700 1 -- 172.21.15.134:0/3541758129 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5280005040 con 0x7f52900f9720 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5295489700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 0x7f52900f73d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f528800d970 tx=0x7f5288004e40).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288004020 con 0x7f52900f9720 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f528800f040 con 0x7f52900f9720 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52880036c0 con 0x7f52900f9720 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5290112300 con 0x7f52900f9720 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5290112820 con 0x7f52900f9720 2024-02-10T08:22:41.293 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529005fec0 con 0x7f52900f9720 2024-02-10T08:22:41.294 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.287+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 14) v1 ==== 77064+0+0 (secure 0 0 0) 0x7f5288003af0 con 0x7f52900f9720 2024-02-10T08:22:41.295 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.291+0000 7f5285ffb700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5278061250 0x7f5278063710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:41.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.291+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7063+0+0 (secure 0 0 0) 0x7f528807c6d0 con 0x7f52900f9720 2024-02-10T08:22:41.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.291+0000 7f5294c88700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5278061250 0x7f5278063710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:41.296 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.291+0000 7f5294c88700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5278061250 0x7f5278063710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5280004840 tx=0x7f5280009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:41.304 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.299+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f528804c590 con 0x7f52900f9720 2024-02-10T08:22:41.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:40 smithi067 bash[17385]: audit 2024-02-10T08:22:39.835039+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T08:22:41.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:40 smithi067 bash[17385]: cluster 2024-02-10T08:22:39.835249+0000 mon.a (mon.0) 566 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-10T08:22:41.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:40 smithi067 bash[24488]: audit 2024-02-10T08:22:39.835039+0000 mon.a (mon.0) 565 : audit [INF] from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T08:22:41.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:40 smithi067 bash[24488]: cluster 2024-02-10T08:22:39.835249+0000 mon.a (mon.0) 566 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-10T08:22:41.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.495+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi134=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f5290059270 con 0x7f5278061250 2024-02-10T08:22:41.515 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5285ffb700 1 -- 172.21.15.134:0/3541758129 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f5290059270 con 0x7f5278061250 2024-02-10T08:22:41.516 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled iscsi.iscsi update... 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5278061250 msgr2=0x7f5278063710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5278061250 0x7f5278063710 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5280004840 tx=0x7f5280009cb0).stop 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 msgr2=0x7f52900f73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 0x7f52900f73d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f528800d970 tx=0x7f5288004e40).stop 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 shutdown_connections 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f5278061250 0x7f5278063710 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900f9720 0x7f52900f73d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52900f5e70 0x7f52900f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 --2- 172.21.15.134:0/3541758129 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52900f6810 0x7f52900f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:41.519 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 >> 172.21.15.134:0/3541758129 conn(0x7f52900f1970 msgr2=0x7f52900fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:41.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 shutdown_connections 2024-02-10T08:22:41.520 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:41.511+0000 7f5296eec700 1 -- 172.21.15.134:0/3541758129 wait complete. 2024-02-10T08:22:41.913 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: cluster 2024-02-10T08:22:40.888146+0000 mon.a (mon.0) 567 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-10T08:22:41.913 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: audit 2024-02-10T08:22:40.893385+0000 mon.a (mon.0) 568 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: audit 2024-02-10T08:22:40.897256+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.67:0/4255108674' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: audit 2024-02-10T08:22:40.903403+0000 mon.a (mon.0) 569 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: cluster 2024-02-10T08:22:40.927968+0000 mgr.y (mgr.14146) 295 : cluster [DBG] pgmap v286: 132 pgs: 14 creating+peering, 65 active+clean, 53 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 510 B/s rd, 255 B/s wr, 0 op/s 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: audit 2024-02-10T08:22:41.514028+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: audit 2024-02-10T08:22:41.893734+0000 mon.a (mon.0) 571 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: audit 2024-02-10T08:22:41.893927+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T08:22:41.914 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:41 smithi134 bash[18722]: cluster 2024-02-10T08:22:41.894493+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-10T08:22:42.237 DEBUG:teuthology.orchestra.run.smithi134:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@iscsi.iscsi.a.service 2024-02-10T08:22:42.239 INFO:tasks.cephadm:Adding prometheus.a on smithi134 2024-02-10T08:22:42.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply prometheus '1;smithi134=a' 2024-02-10T08:22:42.361 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:22:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: cluster 2024-02-10T08:22:40.888146+0000 mon.a (mon.0) 567 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-10T08:22:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: audit 2024-02-10T08:22:40.893385+0000 mon.a (mon.0) 568 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: audit 2024-02-10T08:22:40.897256+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.67:0/4255108674' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: audit 2024-02-10T08:22:40.903403+0000 mon.a (mon.0) 569 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: cluster 2024-02-10T08:22:40.927968+0000 mgr.y (mgr.14146) 295 : cluster [DBG] pgmap v286: 132 pgs: 14 creating+peering, 65 active+clean, 53 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 510 B/s rd, 255 B/s wr, 0 op/s 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: audit 2024-02-10T08:22:41.514028+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: audit 2024-02-10T08:22:41.893734+0000 mon.a (mon.0) 571 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: audit 2024-02-10T08:22:41.893927+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[24488]: cluster 2024-02-10T08:22:41.894493+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: cluster 2024-02-10T08:22:40.888146+0000 mon.a (mon.0) 567 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: audit 2024-02-10T08:22:40.893385+0000 mon.a (mon.0) 568 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: audit 2024-02-10T08:22:40.897256+0000 mon.c (mon.2) 16 : audit [INF] from='client.? 172.21.15.67:0/4255108674' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: audit 2024-02-10T08:22:40.903403+0000 mon.a (mon.0) 569 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: cluster 2024-02-10T08:22:40.927968+0000 mgr.y (mgr.14146) 295 : cluster [DBG] pgmap v286: 132 pgs: 14 creating+peering, 65 active+clean, 53 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 510 B/s rd, 255 B/s wr, 0 op/s 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: audit 2024-02-10T08:22:41.514028+0000 mon.a (mon.0) 570 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: audit 2024-02-10T08:22:41.893734+0000 mon.a (mon.0) 571 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T08:22:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: audit 2024-02-10T08:22:41.893927+0000 mon.a (mon.0) 572 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T08:22:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:41 smithi067 bash[17385]: cluster 2024-02-10T08:22:41.894493+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: audit 2024-02-10T08:22:41.501957+0000 mgr.y (mgr.14146) 296 : audit [DBG] from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi134=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: cephadm 2024-02-10T08:22:41.507175+0000 mgr.y (mgr.14146) 297 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi134=iscsi.a;count:1 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: audit 2024-02-10T08:22:41.898311+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.67:0/4255108674' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: audit 2024-02-10T08:22:41.899108+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: audit 2024-02-10T08:22:41.899514+0000 mon.a (mon.0) 575 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: audit 2024-02-10T08:22:42.897716+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: audit 2024-02-10T08:22:42.897993+0000 mon.a (mon.0) 577 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T08:22:43.221 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:42 smithi134 bash[18722]: cluster 2024-02-10T08:22:42.898193+0000 mon.a (mon.0) 578 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-10T08:22:43.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: audit 2024-02-10T08:22:41.501957+0000 mgr.y (mgr.14146) 296 : audit [DBG] from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi134=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:43.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: cephadm 2024-02-10T08:22:41.507175+0000 mgr.y (mgr.14146) 297 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi134=iscsi.a;count:1 2024-02-10T08:22:43.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: audit 2024-02-10T08:22:41.898311+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.67:0/4255108674' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: audit 2024-02-10T08:22:41.899108+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: audit 2024-02-10T08:22:41.899514+0000 mon.a (mon.0) 575 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: audit 2024-02-10T08:22:42.897716+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: audit 2024-02-10T08:22:42.897993+0000 mon.a (mon.0) 577 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[17385]: cluster 2024-02-10T08:22:42.898193+0000 mon.a (mon.0) 578 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: audit 2024-02-10T08:22:41.501957+0000 mgr.y (mgr.14146) 296 : audit [DBG] from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi134=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: cephadm 2024-02-10T08:22:41.507175+0000 mgr.y (mgr.14146) 297 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi134=iscsi.a;count:1 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: audit 2024-02-10T08:22:41.898311+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.67:0/4255108674' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: audit 2024-02-10T08:22:41.899108+0000 mon.a (mon.0) 574 : audit [INF] from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: audit 2024-02-10T08:22:41.899514+0000 mon.a (mon.0) 575 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: audit 2024-02-10T08:22:42.897716+0000 mon.a (mon.0) 576 : audit [INF] from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T08:22:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: audit 2024-02-10T08:22:42.897993+0000 mon.a (mon.0) 577 : audit [INF] from='client.? 172.21.15.67:0/3542008000' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T08:22:43.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:42 smithi067 bash[24488]: cluster 2024-02-10T08:22:42.898193+0000 mon.a (mon.0) 578 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-10T08:22:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: cluster 2024-02-10T08:22:42.928596+0000 mgr.y (mgr.14146) 298 : cluster [DBG] pgmap v289: 132 pgs: 16 creating+peering, 102 active+clean, 14 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T08:22:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: audit 2024-02-10T08:22:43.122343+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: audit 2024-02-10T08:22:43.132818+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: audit 2024-02-10T08:22:43.141069+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: audit 2024-02-10T08:22:43.161083+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T08:22:44.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: audit 2024-02-10T08:22:43.165362+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T08:22:44.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:43 smithi134 bash[18722]: audit 2024-02-10T08:22:43.186258+0000 mon.a (mon.0) 584 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: cluster 2024-02-10T08:22:42.928596+0000 mgr.y (mgr.14146) 298 : cluster [DBG] pgmap v289: 132 pgs: 16 creating+peering, 102 active+clean, 14 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T08:22:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: audit 2024-02-10T08:22:43.122343+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: audit 2024-02-10T08:22:43.132818+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: audit 2024-02-10T08:22:43.141069+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: audit 2024-02-10T08:22:43.161083+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: audit 2024-02-10T08:22:43.165362+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[24488]: audit 2024-02-10T08:22:43.186258+0000 mon.a (mon.0) 584 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: cluster 2024-02-10T08:22:42.928596+0000 mgr.y (mgr.14146) 298 : cluster [DBG] pgmap v289: 132 pgs: 16 creating+peering, 102 active+clean, 14 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 3 op/s 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: audit 2024-02-10T08:22:43.122343+0000 mon.a (mon.0) 579 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: audit 2024-02-10T08:22:43.132818+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: audit 2024-02-10T08:22:43.141069+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: audit 2024-02-10T08:22:43.161083+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: audit 2024-02-10T08:22:43.165362+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-10T08:22:44.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:43 smithi067 bash[17385]: audit 2024-02-10T08:22:43.186258+0000 mon.a (mon.0) 584 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:44 smithi134 bash[18722]: cephadm 2024-02-10T08:22:43.147519+0000 mgr.y (mgr.14146) 299 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-10T08:22:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:44 smithi134 bash[18722]: cephadm 2024-02-10T08:22:43.187254+0000 mgr.y (mgr.14146) 300 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi134 2024-02-10T08:22:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:44 smithi067 bash[17385]: cephadm 2024-02-10T08:22:43.147519+0000 mgr.y (mgr.14146) 299 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-10T08:22:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:44 smithi067 bash[17385]: cephadm 2024-02-10T08:22:43.187254+0000 mgr.y (mgr.14146) 300 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi134 2024-02-10T08:22:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:44 smithi067 bash[24488]: cephadm 2024-02-10T08:22:43.147519+0000 mgr.y (mgr.14146) 299 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-10T08:22:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:44 smithi067 bash[24488]: cephadm 2024-02-10T08:22:43.187254+0000 mgr.y (mgr.14146) 300 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi134 2024-02-10T08:22:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 -- 172.21.15.134:0/1431986062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 msgr2=0x7faaac0f65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 --2- 172.21.15.134:0/1431986062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 0x7faaac0f65b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7faaa80048a0 tx=0x7faaa8009cb0).stop 2024-02-10T08:22:46.108 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 -- 172.21.15.134:0/1431986062 shutdown_connections 2024-02-10T08:22:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 --2- 172.21.15.134:0/1431986062 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac0f7d80 0x7faaac0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.109 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 --2- 172.21.15.134:0/1431986062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 0x7faaac0f65b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 --2- 172.21.15.134:0/1431986062 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7430 0x7faaac0f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 -- 172.21.15.134:0/1431986062 >> 172.21.15.134:0/1431986062 conn(0x7faaac0f1970 msgr2=0x7faaac0f3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:46.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 -- 172.21.15.134:0/1431986062 shutdown_connections 2024-02-10T08:22:46.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 -- 172.21.15.134:0/1431986062 wait complete. 2024-02-10T08:22:46.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 Processor -- start 2024-02-10T08:22:46.112 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.102+0000 7faab44e4700 1 -- start start 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 0x7faaac107fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac0f7430 0x7faaac1084d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 0x7faaac10f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10fa50 con 0x7faaac0f7430 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10fb90 con 0x7faaac0f6190 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac10fcd0 con 0x7faaac0f7d80 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 0x7faaac10f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 0x7faaac10f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.134:43070/0 (socket says 172.21.15.134:43070) 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 -- 172.21.15.134:0/2673823702 learned_addr learned my addr 172.21.15.134:0/2673823702 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab1a7f700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac0f7430 0x7faaac1084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:46.113 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2280700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 0x7faaac107fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 -- 172.21.15.134:0/2673823702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 msgr2=0x7faaac107fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 0x7faaac107fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 -- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac0f7430 msgr2=0x7faaac1084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac0f7430 0x7faaac1084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 -- 172.21.15.134:0/2673823702 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8005040 con 0x7faaac0f7d80 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab2a81700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 0x7faaac10f560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7faaa400ec20 tx=0x7faaa400b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:46.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4009950 con 0x7faaac0f7d80 2024-02-10T08:22:46.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac10ff50 con 0x7faaac0f7d80 2024-02-10T08:22:46.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7faaa400f040 con 0x7faaac0f7d80 2024-02-10T08:22:46.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac110430 con 0x7faaac0f7d80 2024-02-10T08:22:46.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.106+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007d20 con 0x7faaac0f7d80 2024-02-10T08:22:46.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.110+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac102810 con 0x7faaac0f7d80 2024-02-10T08:22:46.115 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.110+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 15) v1 ==== 77115+0+0 (secure 0 0 0) 0x7faaa40092e0 con 0x7faaac0f7d80 2024-02-10T08:22:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.110+0000 7faaa37fe700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7faa980612e0 0x7faa980637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.110+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7090+0+0 (secure 0 0 0) 0x7faaa407cb10 con 0x7faaac0f7d80 2024-02-10T08:22:46.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.110+0000 7faab2280700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7faa980612e0 0x7faa980637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:46.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.110+0000 7faab2280700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7faa980612e0 0x7faa980637a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7faaa8009f20 tx=0x7faaa8009cb0).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:46.127 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.126+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faaa4049f60 con 0x7faaac0f7d80 2024-02-10T08:22:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:45 smithi134 bash[18722]: cluster 2024-02-10T08:22:44.929564+0000 mgr.y (mgr.14146) 301 : cluster [DBG] pgmap v290: 132 pgs: 9 creating+peering, 114 active+clean, 9 unknown; 2.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 2.6 KiB/s wr, 82 op/s 2024-02-10T08:22:46.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.314+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}) v1 -- 0x7faaac0fdd20 con 0x7faa980612e0 2024-02-10T08:22:46.336 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.330+0000 7faaa37fe700 1 -- 172.21.15.134:0/2673823702 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7faaac0fdd20 con 0x7faa980612e0 2024-02-10T08:22:46.336 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled prometheus update... 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7faa980612e0 msgr2=0x7faa980637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7faa980612e0 0x7faa980637a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7faaa8009f20 tx=0x7faaa8009cb0).stop 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 msgr2=0x7faaac10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 0x7faaac10f560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7faaa400ec20 tx=0x7faaa400b040).stop 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 shutdown_connections 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7faa980612e0 0x7faa980637a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faaac0f7d80 0x7faaac10f560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaac0f6190 0x7faaac107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 --2- 172.21.15.134:0/2673823702 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faaac0f7430 0x7faaac1084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:46.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 >> 172.21.15.134:0/2673823702 conn(0x7faaac0f1970 msgr2=0x7faaac0fae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:46.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 shutdown_connections 2024-02-10T08:22:46.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:46.334+0000 7faab44e4700 1 -- 172.21.15.134:0/2673823702 wait complete. 2024-02-10T08:22:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:45 smithi067 bash[17385]: cluster 2024-02-10T08:22:44.929564+0000 mgr.y (mgr.14146) 301 : cluster [DBG] pgmap v290: 132 pgs: 9 creating+peering, 114 active+clean, 9 unknown; 2.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 2.6 KiB/s wr, 82 op/s 2024-02-10T08:22:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:45 smithi067 bash[24488]: cluster 2024-02-10T08:22:44.929564+0000 mgr.y (mgr.14146) 301 : cluster [DBG] pgmap v290: 132 pgs: 9 creating+peering, 114 active+clean, 9 unknown; 2.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 34 KiB/s rd, 2.6 KiB/s wr, 82 op/s 2024-02-10T08:22:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: cluster 2024-02-10T08:22:45.946481+0000 mon.a (mon.0) 585 : cluster [DBG] mgrmap e15: y(active, since 8m), standbys: x 2024-02-10T08:22:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: audit 2024-02-10T08:22:46.335424+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: audit 2024-02-10T08:22:46.850863+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: audit 2024-02-10T08:22:46.853168+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[17385]: cluster 2024-02-10T08:22:45.946481+0000 mon.a (mon.0) 585 : cluster [DBG] mgrmap e15: y(active, since 8m), standbys: x 2024-02-10T08:22:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[17385]: audit 2024-02-10T08:22:46.335424+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[17385]: audit 2024-02-10T08:22:46.850863+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[17385]: audit 2024-02-10T08:22:46.853168+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[24488]: cluster 2024-02-10T08:22:45.946481+0000 mon.a (mon.0) 585 : cluster [DBG] mgrmap e15: y(active, since 8m), standbys: x 2024-02-10T08:22:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[24488]: audit 2024-02-10T08:22:46.335424+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[24488]: audit 2024-02-10T08:22:46.850863+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:46 smithi067 bash[24488]: audit 2024-02-10T08:22:46.853168+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:22:47.609 DEBUG:teuthology.orchestra.run.smithi134:prometheus.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@prometheus.a.service 2024-02-10T08:22:47.612 INFO:tasks.cephadm:Adding node-exporter.a on smithi067 2024-02-10T08:22:47.612 INFO:tasks.cephadm:Adding node-exporter.b on smithi134 2024-02-10T08:22:47.612 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply node-exporter '2;smithi067=a;smithi134=b' 2024-02-10T08:22:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: audit 2024-02-10T08:22:46.324406+0000 mgr.y (mgr.14146) 302 : audit [DBG] from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: cephadm 2024-02-10T08:22:46.328708+0000 mgr.y (mgr.14146) 303 : cephadm [INF] Saving service prometheus spec with placement smithi134=a;count:1 2024-02-10T08:22:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: cluster 2024-02-10T08:22:46.930728+0000 mgr.y (mgr.14146) 304 : cluster [DBG] pgmap v291: 132 pgs: 132 active+clean; 4.5 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 4.8 KiB/s wr, 148 op/s 2024-02-10T08:22:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:47 smithi134 bash[18722]: audit 2024-02-10T08:22:46.970393+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[17385]: audit 2024-02-10T08:22:46.324406+0000 mgr.y (mgr.14146) 302 : audit [DBG] from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[17385]: cephadm 2024-02-10T08:22:46.328708+0000 mgr.y (mgr.14146) 303 : cephadm [INF] Saving service prometheus spec with placement smithi134=a;count:1 2024-02-10T08:22:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[17385]: cluster 2024-02-10T08:22:46.930728+0000 mgr.y (mgr.14146) 304 : cluster [DBG] pgmap v291: 132 pgs: 132 active+clean; 4.5 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 4.8 KiB/s wr, 148 op/s 2024-02-10T08:22:48.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[17385]: audit 2024-02-10T08:22:46.970393+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[24488]: audit 2024-02-10T08:22:46.324406+0000 mgr.y (mgr.14146) 302 : audit [DBG] from='client.24428 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[24488]: cephadm 2024-02-10T08:22:46.328708+0000 mgr.y (mgr.14146) 303 : cephadm [INF] Saving service prometheus spec with placement smithi134=a;count:1 2024-02-10T08:22:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[24488]: cluster 2024-02-10T08:22:46.930728+0000 mgr.y (mgr.14146) 304 : cluster [DBG] pgmap v291: 132 pgs: 132 active+clean; 4.5 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 4.8 KiB/s wr, 148 op/s 2024-02-10T08:22:48.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:47 smithi067 bash[24488]: audit 2024-02-10T08:22:46.970393+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:48.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:22:49.651 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:49 smithi134 bash[39473]: log file path now is '/var/log/tcmu-runner.log' 2024-02-10T08:22:49.979 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:49 smithi134 bash[39502]: debug Started the configuration object watcher 2024-02-10T08:22:49.979 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:49 smithi134 bash[39502]: debug Checking for config object changes every 1s 2024-02-10T08:22:49.979 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:49 smithi134 bash[39502]: debug Processing osd blocklist entries for this node 2024-02-10T08:22:50.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:49 smithi134 bash[18722]: cluster 2024-02-10T08:22:48.931691+0000 mgr.y (mgr.14146) 305 : cluster [DBG] pgmap v292: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-02-10T08:22:50.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:50 smithi134 bash[39502]: debug Reading the configuration object to update local LIO configuration 2024-02-10T08:22:50.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:50 smithi134 bash[39502]: debug Configuration does not have an entry for this host(smithi134.front.sepia.ceph.com) - nothing to define to LIO 2024-02-10T08:22:50.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:22:50 smithi134 bash[39502]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-10T08:22:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:49 smithi067 bash[17385]: cluster 2024-02-10T08:22:48.931691+0000 mgr.y (mgr.14146) 305 : cluster [DBG] pgmap v292: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-02-10T08:22:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:49 smithi067 bash[24488]: cluster 2024-02-10T08:22:48.931691+0000 mgr.y (mgr.14146) 305 : cluster [DBG] pgmap v292: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-02-10T08:22:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:50 smithi134 bash[18722]: audit 2024-02-10T08:22:50.056052+0000 mon.b (mon.1) 21 : audit [DBG] from='client.? 172.21.15.134:0/2578140463' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T08:22:51.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:50 smithi067 bash[17385]: audit 2024-02-10T08:22:50.056052+0000 mon.b (mon.1) 21 : audit [DBG] from='client.? 172.21.15.134:0/2578140463' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T08:22:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:50 smithi067 bash[24488]: audit 2024-02-10T08:22:50.056052+0000 mon.b (mon.1) 21 : audit [DBG] from='client.? 172.21.15.134:0/2578140463' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 -- 172.21.15.134:0/2720726633 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0fd730 msgr2=0x7f3d0c0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 --2- 172.21.15.134:0/2720726633 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0fd730 0x7f3d0c0ffbc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f3d04009c30 tx=0x7f3d04004c50).stop 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 -- 172.21.15.134:0/2720726633 shutdown_connections 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 --2- 172.21.15.134:0/2720726633 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d0c0f5eb0 0x7f3d0c0f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 --2- 172.21.15.134:0/2720726633 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0fd730 0x7f3d0c0ffbc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 --2- 172.21.15.134:0/2720726633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0f6850 0x7f3d0c0fd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 -- 172.21.15.134:0/2720726633 >> 172.21.15.134:0/2720726633 conn(0x7f3d0c0f19d0 msgr2=0x7f3d0c0f3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 -- 172.21.15.134:0/2720726633 shutdown_connections 2024-02-10T08:22:51.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.494+0000 7f3d14563700 1 -- 172.21.15.134:0/2720726633 wait complete. 2024-02-10T08:22:51.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 Processor -- start 2024-02-10T08:22:51.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- start start 2024-02-10T08:22:51.499 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0f5eb0 0x7f3d0c0f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:51.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d0c0f6850 0x7f3d0c0f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:51.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 0x7f3d0c0f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:51.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0f7a30 con 0x7f3d0c0f6850 2024-02-10T08:22:51.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0f7ba0 con 0x7f3d0c0fd730 2024-02-10T08:22:51.500 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c0f7d60 con 0x7f3d0c0f5eb0 2024-02-10T08:22:51.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 0x7f3d0c0f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:51.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 0x7f3d0c0f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:58130/0 (socket says 172.21.15.134:58130) 2024-02-10T08:22:51.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 -- 172.21.15.134:0/1762900203 learned_addr learned my addr 172.21.15.134:0/1762900203 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:51.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d122ff700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0f5eb0 0x7f3d0c0f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:51.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d11afe700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d0c0f6850 0x7f3d0c0f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:51.501 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 -- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0f5eb0 msgr2=0x7f3d0c0f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0f5eb0 0x7f3d0c0f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 -- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d0c0f6850 msgr2=0x7f3d0c0f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d0c0f6850 0x7f3d0c0f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 -- 172.21.15.134:0/1762900203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040097c0 con 0x7f3d0c0fd730 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d12b00700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 0x7f3d0c0f7410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3d040095f0 tx=0x7f3d04004c50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0400e030 con 0x7f3d0c0fd730 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c1120d0 con 0x7f3d0c0fd730 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c1124d0 con 0x7f3d0c0fd730 2024-02-10T08:22:51.502 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f3d04004ed0 con 0x7f3d0c0fd730 2024-02-10T08:22:51.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d04018070 con 0x7f3d0c0fd730 2024-02-10T08:22:51.503 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.498+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf8004f50 con 0x7f3d0c0fd730 2024-02-10T08:22:51.504 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.502+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) v1 ==== 77115+0+0 (secure 0 0 0) 0x7f3d04007b20 con 0x7f3d0c0fd730 2024-02-10T08:22:51.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.502+0000 7f3d0b7fe700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3cf40612e0 0x7f3cf40637a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:51.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.502+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7090+0+0 (secure 0 0 0) 0x7f3d0407da40 con 0x7f3d0c0fd730 2024-02-10T08:22:51.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.502+0000 7f3d122ff700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3cf40612e0 0x7f3cf40637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:51.506 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.502+0000 7f3d122ff700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3cf40612e0 0x7f3cf40637a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f3cfc0048b0 tx=0x7f3cfc00b040).ready entity=mgr.14146 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:51.509 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.506+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3d0404d710 con 0x7f3d0c0fd730 2024-02-10T08:22:51.707 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.702+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi067=a;smithi134=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cf8000bf0 con 0x7f3cf40612e0 2024-02-10T08:22:51.719 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.714+0000 7f3d0b7fe700 1 -- 172.21.15.134:0/1762900203 <== mgr.14146 v2:172.21.15.67:6800/4022632212 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3cf8000bf0 con 0x7f3cf40612e0 2024-02-10T08:22:51.719 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled node-exporter update... 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3cf40612e0 msgr2=0x7f3cf40637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3cf40612e0 0x7f3cf40637a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f3cfc0048b0 tx=0x7f3cfc00b040).stop 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 msgr2=0x7f3d0c0f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 0x7f3d0c0f7410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3d040095f0 tx=0x7f3d04004c50).stop 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 shutdown_connections 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f3cf40612e0 0x7f3cf40637a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d0c0fd730 0x7f3d0c0f7410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.722 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3d0c0f5eb0 0x7f3d0c0f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 --2- 172.21.15.134:0/1762900203 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3d0c0f6850 0x7f3d0c0f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 >> 172.21.15.134:0/1762900203 conn(0x7f3d0c0f19d0 msgr2=0x7f3d0c0f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 shutdown_connections 2024-02-10T08:22:51.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:51.718+0000 7f3d14563700 1 -- 172.21.15.134:0/1762900203 wait complete. 2024-02-10T08:22:52.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:51 smithi134 bash[18722]: cluster 2024-02-10T08:22:50.932813+0000 mgr.y (mgr.14146) 306 : cluster [DBG] pgmap v293: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 124 op/s 2024-02-10T08:22:52.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:51 smithi134 bash[18722]: audit 2024-02-10T08:22:51.718083+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:52.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:51 smithi067 bash[17385]: cluster 2024-02-10T08:22:50.932813+0000 mgr.y (mgr.14146) 306 : cluster [DBG] pgmap v293: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 124 op/s 2024-02-10T08:22:52.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:51 smithi067 bash[17385]: audit 2024-02-10T08:22:51.718083+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:51 smithi067 bash[24488]: cluster 2024-02-10T08:22:50.932813+0000 mgr.y (mgr.14146) 306 : cluster [DBG] pgmap v293: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 3.9 KiB/s wr, 124 op/s 2024-02-10T08:22:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:51 smithi067 bash[24488]: audit 2024-02-10T08:22:51.718083+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:52.466 DEBUG:teuthology.orchestra.run.smithi067:node-exporter.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@node-exporter.a.service 2024-02-10T08:22:52.469 DEBUG:teuthology.orchestra.run.smithi134:node-exporter.b> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@node-exporter.b.service 2024-02-10T08:22:52.473 INFO:tasks.cephadm:Adding alertmanager.a on smithi067 2024-02-10T08:22:52.474 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply alertmanager '1;smithi067=a' 2024-02-10T08:22:52.535 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:22:52.602 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:22:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: audit 2024-02-10T08:22:51.708878+0000 mgr.y (mgr.14146) 307 : audit [DBG] from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi067=a;smithi134=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: cephadm 2024-02-10T08:22:51.712730+0000 mgr.y (mgr.14146) 308 : cephadm [INF] Saving service node-exporter spec with placement smithi067=a;smithi134=b;count:2 2024-02-10T08:22:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: audit 2024-02-10T08:22:52.272194+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: audit 2024-02-10T08:22:52.565011+0000 mon.a (mon.0) 592 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: audit 2024-02-10T08:22:52.567094+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: audit 2024-02-10T08:22:52.576390+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17385]: audit 2024-02-10T08:22:52.585536+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: audit 2024-02-10T08:22:51.708878+0000 mgr.y (mgr.14146) 307 : audit [DBG] from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi067=a;smithi134=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: cephadm 2024-02-10T08:22:51.712730+0000 mgr.y (mgr.14146) 308 : cephadm [INF] Saving service node-exporter spec with placement smithi067=a;smithi134=b;count:2 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: audit 2024-02-10T08:22:52.272194+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: audit 2024-02-10T08:22:52.565011+0000 mon.a (mon.0) 592 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: audit 2024-02-10T08:22:52.567094+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: audit 2024-02-10T08:22:52.576390+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[24488]: audit 2024-02-10T08:22:52.585536+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T08:22:53.493 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: audit 2024-02-10T08:22:51.708878+0000 mgr.y (mgr.14146) 307 : audit [DBG] from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi067=a;smithi134=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:22:53.493 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: cephadm 2024-02-10T08:22:51.712730+0000 mgr.y (mgr.14146) 308 : cephadm [INF] Saving service node-exporter spec with placement smithi067=a;smithi134=b;count:2 2024-02-10T08:22:53.494 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: audit 2024-02-10T08:22:52.272194+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:53.494 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: audit 2024-02-10T08:22:52.565011+0000 mon.a (mon.0) 592 : audit [DBG] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:22:53.494 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: audit 2024-02-10T08:22:52.567094+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:22:53.494 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: audit 2024-02-10T08:22:52.576390+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' 2024-02-10T08:22:53.494 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[18722]: audit 2024-02-10T08:22:52.585536+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T08:22:53.777 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[19742]: ignoring --setuser ceph since I am not root 2024-02-10T08:22:53.777 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[19742]: ignoring --setgroup ceph since I am not root 2024-02-10T08:22:53.777 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[19742]: debug 2024-02-10T08:22:53.630+0000 7f270ca88700 1 -- 172.21.15.134:0/2500796776 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f3c54304e0 con 0x55f3c5524c00 2024-02-10T08:22:53.911 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17647]: ignoring --setuser ceph since I am not root 2024-02-10T08:22:53.911 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17647]: ignoring --setgroup ceph since I am not root 2024-02-10T08:22:53.911 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17647]: debug 2024-02-10T08:22:53.632+0000 7f34b4ca7700 1 -- 172.21.15.67:0/3849828267 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd7b98c4e0 con 0x55cd7ba81000 2024-02-10T08:22:53.911 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17647]: debug 2024-02-10T08:22:53.792+0000 7f34c4e85680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T08:22:54.108 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[19742]: debug 2024-02-10T08:22:53.794+0000 7f271cc66680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T08:22:54.109 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:53 smithi134 bash[19742]: debug 2024-02-10T08:22:53.902+0000 7f271cc66680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T08:22:54.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:53 smithi067 bash[17647]: debug 2024-02-10T08:22:53.904+0000 7f34c4e85680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T08:22:54.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[24488]: cluster 2024-02-10T08:22:52.933957+0000 mgr.y (mgr.14146) 309 : cluster [DBG] pgmap v294: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T08:22:54.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[24488]: audit 2024-02-10T08:22:53.590804+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T08:22:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[24488]: cluster 2024-02-10T08:22:53.590885+0000 mon.a (mon.0) 597 : cluster [DBG] mgrmap e16: y(active, since 8m), standbys: x 2024-02-10T08:22:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[17385]: cluster 2024-02-10T08:22:52.933957+0000 mgr.y (mgr.14146) 309 : cluster [DBG] pgmap v294: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T08:22:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[17385]: audit 2024-02-10T08:22:53.590804+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T08:22:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[17385]: cluster 2024-02-10T08:22:53.590885+0000 mon.a (mon.0) 597 : cluster [DBG] mgrmap e16: y(active, since 8m), standbys: x 2024-02-10T08:22:54.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:54 smithi134 bash[18722]: cluster 2024-02-10T08:22:52.933957+0000 mgr.y (mgr.14146) 309 : cluster [DBG] pgmap v294: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T08:22:54.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:54 smithi134 bash[18722]: audit 2024-02-10T08:22:53.590804+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 172.21.15.67:0/1646746415' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T08:22:54.382 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:22:54 smithi134 bash[18722]: cluster 2024-02-10T08:22:53.590885+0000 mon.a (mon.0) 597 : cluster [DBG] mgrmap e16: y(active, since 8m), standbys: x 2024-02-10T08:22:54.777 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:54 smithi134 bash[19742]: debug 2024-02-10T08:22:54.378+0000 7f271cc66680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T08:22:54.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:54 smithi067 bash[17647]: debug 2024-02-10T08:22:54.400+0000 7f34c4e85680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T08:22:55.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 -- 172.21.15.134:0/2216784340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 msgr2=0x7f80000fa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:55.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 --2- 172.21.15.134:0/2216784340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 0x7f80000fa710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7ff0004870 tx=0x7f7ff0009cb0).stop 2024-02-10T08:22:55.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 -- 172.21.15.134:0/2216784340 shutdown_connections 2024-02-10T08:22:55.407 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 --2- 172.21.15.134:0/2216784340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80000fd720 0x7f80000ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 --2- 172.21.15.134:0/2216784340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 0x7f80000fa710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 --2- 172.21.15.134:0/2216784340 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fac90 0x7f80000fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 -- 172.21.15.134:0/2216784340 >> 172.21.15.134:0/2216784340 conn(0x7f80000f2230 msgr2=0x7f80000f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 -- 172.21.15.134:0/2216784340 shutdown_connections 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.402+0000 7f8006d27700 1 -- 172.21.15.134:0/2216784340 wait complete. 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 Processor -- start 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- start start 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 0x7f8000107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80000fac90 0x7f8000108430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fd720 0x7f800010f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800010fa00 con 0x7f80000fac90 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800010fb40 con 0x7f8000061770 2024-02-10T08:22:55.408 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800010fc80 con 0x7f80000fd720 2024-02-10T08:22:55.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fd720 0x7f800010f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:55.409 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8004ac3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 0x7f8000107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80000fac90 0x7f8000108430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fd720 0x7f800010f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.134:56814/0 (socket says 172.21.15.134:56814) 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 -- 172.21.15.134:0/1946803161 learned_addr learned my addr 172.21.15.134:0/1946803161 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 msgr2=0x7f8000107f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8000061770 0x7f8000107f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80000fac90 msgr2=0x7f8000108430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80000fac90 0x7f8000108430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 -- 172.21.15.134:0/1946803161 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0005040 con 0x7f80000fd720 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f80052c4700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fd720 0x7f800010f510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7ff800db50 tx=0x7f7ff8004f00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800b750 con 0x7f80000fd720 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f7ff800f040 con 0x7f80000fd720 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8013930 con 0x7f80000fd720 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80000f6610 con 0x7f80000fd720 2024-02-10T08:22:55.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80000f6b00 con 0x7f80000fd720 2024-02-10T08:22:55.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8000102750 con 0x7f80000fd720 2024-02-10T08:22:55.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.406+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 16) v1 ==== 77129+0+0 (secure 0 0 0) 0x7f7ff800bac0 con 0x7f80000fd720 2024-02-10T08:22:55.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.410+0000 7f7ffdffb700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:22:55.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.410+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7090+0+0 (secure 0 0 0) 0x7f7ff807caf0 con 0x7f80000fd720 2024-02-10T08:22:55.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.410+0000 7f8004ac3700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 msgr2=0x7f7fe80637f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4022632212 2024-02-10T08:22:55.412 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.410+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2024-02-10T08:22:55.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.418+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7ff804c760 con 0x7f80000fd720 2024-02-10T08:22:55.457 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:55 smithi134 bash[19742]: debug 2024-02-10T08:22:55.346+0000 7f271cc66680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T08:22:55.458 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:55 smithi134 bash[19742]: debug 2024-02-10T08:22:55.454+0000 7f271cc66680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T08:22:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.614+0000 7f8004ac3700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 msgr2=0x7f7fe80637f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4022632212 2024-02-10T08:22:55.615 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.614+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2024-02-10T08:22:55.619 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:55.618+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 --> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f80000591c0 con 0x7f7fe8061330 2024-02-10T08:22:55.768 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:55 smithi067 bash[17647]: debug 2024-02-10T08:22:55.400+0000 7f34c4e85680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T08:22:55.768 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:55 smithi067 bash[17647]: debug 2024-02-10T08:22:55.504+0000 7f34c4e85680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T08:22:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:56.018+0000 7f8004ac3700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 msgr2=0x7f7fe80637f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4022632212 2024-02-10T08:22:56.020 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:56.018+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2024-02-10T08:22:56.027 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:55 smithi134 bash[19742]: debug 2024-02-10T08:22:55.730+0000 7f271cc66680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T08:22:56.823 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:56.822+0000 7f8004ac3700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 msgr2=0x7f7fe80637f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4022632212 2024-02-10T08:22:56.824 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:56.822+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2024-02-10T08:22:57.213 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:56 smithi134 bash[19742]: debug 2024-02-10T08:22:56.946+0000 7f271cc66680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T08:22:57.214 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:57 smithi134 bash[19742]: debug 2024-02-10T08:22:57.114+0000 7f271cc66680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:22:57.232 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:55 smithi067 bash[17647]: debug 2024-02-10T08:22:55.764+0000 7f34c4e85680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T08:22:57.232 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:56 smithi067 bash[17647]: debug 2024-02-10T08:22:56.952+0000 7f34c4e85680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T08:22:57.232 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:57 smithi067 bash[17647]: debug 2024-02-10T08:22:57.124+0000 7f34c4e85680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:22:57.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:57 smithi134 bash[19742]: debug 2024-02-10T08:22:57.210+0000 7f271cc66680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T08:22:57.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:57 smithi134 bash[19742]: debug 2024-02-10T08:22:57.322+0000 7f271cc66680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T08:22:57.582 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:57 smithi067 bash[17647]: debug 2024-02-10T08:22:57.228+0000 7f34c4e85680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T08:22:57.582 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:57 smithi067 bash[17647]: debug 2024-02-10T08:22:57.348+0000 7f34c4e85680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T08:22:57.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:57 smithi067 bash[17647]: debug 2024-02-10T08:22:57.576+0000 7f34c4e85680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T08:22:57.911 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:57 smithi134 bash[19742]: debug 2024-02-10T08:22:57.542+0000 7f271cc66680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T08:22:58.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:57 smithi134 bash[19742]: debug 2024-02-10T08:22:57.910+0000 7f271cc66680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T08:22:58.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:58 smithi134 bash[19742]: debug 2024-02-10T08:22:58.030+0000 7f271cc66680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T08:22:58.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:57 smithi067 bash[17647]: debug 2024-02-10T08:22:57.972+0000 7f34c4e85680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T08:22:58.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:58 smithi067 bash[17647]: debug 2024-02-10T08:22:58.100+0000 7f34c4e85680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T08:22:58.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:58.426+0000 7f8004ac3700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 msgr2=0x7f7fe80637f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.67:6800/4022632212 2024-02-10T08:22:58.427 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:22:58.426+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2024-02-10T08:22:59.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:58 smithi134 bash[19742]: debug 2024-02-10T08:22:58.898+0000 7f271cc66680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T08:22:59.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:59 smithi134 bash[19742]: debug 2024-02-10T08:22:59.002+0000 7f271cc66680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T08:22:59.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:59 smithi134 bash[19742]: debug 2024-02-10T08:22:59.118+0000 7f271cc66680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T08:22:59.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:59 smithi067 bash[17647]: debug 2024-02-10T08:22:59.008+0000 7f34c4e85680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T08:22:59.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:59 smithi067 bash[17647]: debug 2024-02-10T08:22:59.120+0000 7f34c4e85680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T08:22:59.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:59 smithi067 bash[17647]: debug 2024-02-10T08:22:59.244+0000 7f34c4e85680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T08:22:59.630 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:59 smithi134 bash[19742]: debug 2024-02-10T08:22:59.334+0000 7f271cc66680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T08:22:59.788 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:59 smithi067 bash[17647]: debug 2024-02-10T08:22:59.484+0000 7f34c4e85680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T08:22:59.966 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:59 smithi134 bash[19742]: debug 2024-02-10T08:22:59.626+0000 7f271cc66680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T08:22:59.966 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:59 smithi134 bash[19742]: debug 2024-02-10T08:22:59.786+0000 7f271cc66680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T08:23:00.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:59 smithi067 bash[17647]: debug 2024-02-10T08:22:59.784+0000 7f34c4e85680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T08:23:00.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:22:59 smithi067 bash[17647]: debug 2024-02-10T08:22:59.960+0000 7f34c4e85680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T08:23:00.227 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:22:59 smithi134 bash[19742]: debug 2024-02-10T08:22:59.962+0000 7f271cc66680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:23:00.421 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:00.418+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mgrmap(e 17) v1 ==== 77129+0+0 (secure 0 0 0) 0x7f7ff807a320 con 0x7f80000fd720 2024-02-10T08:23:00.422 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: debug 2024-02-10T08:23:00.144+0000 7f34c4e85680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T08:23:00.422 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: debug 2024-02-10T08:23:00.368+0000 7f34c4e85680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T08:23:00.424 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.358561+0000 mon.a (mon.0) 598 : cluster [DBG] Standby manager daemon x restarted 2024-02-10T08:23:00.424 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.358754+0000 mon.a (mon.0) 599 : cluster [DBG] Standby manager daemon x started 2024-02-10T08:23:00.424 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[24488]: audit 2024-02-10T08:23:00.360723+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T08:23:00.424 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[24488]: audit 2024-02-10T08:23:00.362081+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T08:23:00.424 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[24488]: audit 2024-02-10T08:23:00.367108+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T08:23:00.424 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[24488]: audit 2024-02-10T08:23:00.367762+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T08:23:00.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:00.518+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 8 ==== mgrmap(e 18) v1 ==== 76859+0+0 (secure 0 0 0) 0x7f7ff807a6a0 con 0x7f80000fd720 2024-02-10T08:23:00.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:00.518+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 msgr2=0x7f7fe80637f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-10T08:23:00.523 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:00.518+0000 7f7ffdffb700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4022632212,v1:172.21.15.67:6801/4022632212] conn(0x7f7fe8061330 0x7f7fe80637f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: debug 2024-02-10T08:23:00.222+0000 7f271cc66680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: debug 2024-02-10T08:23:00.330+0000 7f271cc66680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: [10/Feb/2024:08:23:00] ENGINE Bus STARTING 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: CherryPy Checker: 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: The Application mounted at '' has an empty config. 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: [10/Feb/2024:08:23:00] ENGINE Serving on http://:::9283 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[19742]: [10/Feb/2024:08:23:00] ENGINE Bus STARTED 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.358561+0000 mon.a (mon.0) 598 : cluster [DBG] Standby manager daemon x restarted 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.358754+0000 mon.a (mon.0) 599 : cluster [DBG] Standby manager daemon x started 2024-02-10T08:23:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[18722]: audit 2024-02-10T08:23:00.360723+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T08:23:00.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[18722]: audit 2024-02-10T08:23:00.362081+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T08:23:00.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[18722]: audit 2024-02-10T08:23:00.367108+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T08:23:00.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:00 smithi134 bash[18722]: audit 2024-02-10T08:23:00.367762+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T08:23:00.760 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: debug 2024-02-10T08:23:00.484+0000 7f34c4e85680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T08:23:00.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.358561+0000 mon.a (mon.0) 598 : cluster [DBG] Standby manager daemon x restarted 2024-02-10T08:23:00.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.358754+0000 mon.a (mon.0) 599 : cluster [DBG] Standby manager daemon x started 2024-02-10T08:23:00.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17385]: audit 2024-02-10T08:23:00.360723+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T08:23:00.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17385]: audit 2024-02-10T08:23:00.362081+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T08:23:00.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17385]: audit 2024-02-10T08:23:00.367108+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T08:23:00.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17385]: audit 2024-02-10T08:23:00.367762+0000 mon.b (mon.1) 25 : audit [DBG] from='mgr.? 172.21.15.134:0/3449967858' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T08:23:01.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: [10/Feb/2024:08:23:00] ENGINE Bus STARTING 2024-02-10T08:23:01.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: Warning: Permanently added '172.21.15.67' (ECDSA) to the list of known hosts. 2024-02-10T08:23:01.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: Warning: Permanently added '172.21.15.134' (ECDSA) to the list of known hosts. 2024-02-10T08:23:01.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: CherryPy Checker: 2024-02-10T08:23:01.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: The Application mounted at '' has an empty config. 2024-02-10T08:23:01.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: [10/Feb/2024:08:23:00] ENGINE Serving on http://:::9283 2024-02-10T08:23:01.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:00 smithi067 bash[17647]: [10/Feb/2024:08:23:00] ENGINE Bus STARTED 2024-02-10T08:23:01.531 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.530+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mon.2 v2:172.21.15.67:3301/0 9 ==== mgrmap(e 19) v1 ==== 76929+0+0 (secure 0 0 0) 0x7f7ff8013d20 con 0x7f80000fd720 2024-02-10T08:23:01.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.530+0000 7f7ffdffb700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7fe8030480 0x7f7fe806cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:01.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.530+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 --> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f80000591c0 con 0x7f7fe8030480 2024-02-10T08:23:01.532 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.530+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7fe8030480 0x7f7fe806cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:01.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.530+0000 7f8004ac3700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7fe8030480 0x7f7fe806cd90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f7ff0009920 tx=0x7f7ff0009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:01.544 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.542+0000 7f7ffdffb700 1 -- 172.21.15.134:0/1946803161 <== mgr.14547 v2:172.21.15.67:6800/4090043441 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f80000591c0 con 0x7f7fe8030480 2024-02-10T08:23:01.545 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled alertmanager update... 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7fe8030480 msgr2=0x7f7fe806cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7fe8030480 0x7f7fe806cd90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f7ff0009920 tx=0x7f7ff0009cb0).stop 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fd720 msgr2=0x7f800010f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 --2- 172.21.15.134:0/1946803161 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80000fd720 0x7f800010f510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7ff800db50 tx=0x7f7ff8004f00).stop 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f80052c4700 1 -- 172.21.15.134:0/1946803161 reap_dead start 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 shutdown_connections 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 >> 172.21.15.134:0/1946803161 conn(0x7f80000f2230 msgr2=0x7f80000f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 shutdown_connections 2024-02-10T08:23:01.548 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:01.546+0000 7f8006d27700 1 -- 172.21.15.134:0/1946803161 wait complete. 2024-02-10T08:23:01.777 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.420318+0000 mon.a (mon.0) 600 : cluster [DBG] mgrmap e17: y(active, since 8m), standbys: x 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.509094+0000 mon.a (mon.0) 601 : cluster [INF] Active manager daemon y restarted 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.509741+0000 mon.a (mon.0) 602 : cluster [INF] Activating manager daemon y 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.522305+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.522892+0000 mon.a (mon.0) 604 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0132743s), standbys: x 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.524303+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.524539+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.524797+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.525877+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T08:23:01.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.526142+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.526402+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.526715+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.527060+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.527398+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.527788+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.528072+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.528293+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.528566+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.528825+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.529029+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.529522+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T08:23:01.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: cluster 2024-02-10T08:23:00.533583+0000 mon.a (mon.0) 605 : cluster [INF] Manager daemon y is now available 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.682865+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.685703+0000 mon.c (mon.2) 35 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.729027+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.763941+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.764456+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.810948+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:23:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:01 smithi134 bash[18722]: audit 2024-02-10T08:23:00.811492+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:23:01.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.420318+0000 mon.a (mon.0) 600 : cluster [DBG] mgrmap e17: y(active, since 8m), standbys: x 2024-02-10T08:23:01.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.509094+0000 mon.a (mon.0) 601 : cluster [INF] Active manager daemon y restarted 2024-02-10T08:23:01.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.509741+0000 mon.a (mon.0) 602 : cluster [INF] Activating manager daemon y 2024-02-10T08:23:01.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.522305+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-10T08:23:01.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.522892+0000 mon.a (mon.0) 604 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0132743s), standbys: x 2024-02-10T08:23:01.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.524303+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:23:01.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.524539+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:23:01.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.524797+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:23:01.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.525877+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T08:23:01.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.526142+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T08:23:01.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.526402+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:23:01.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.526715+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:23:01.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.527060+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:23:01.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.527398+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:23:01.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.527788+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:23:01.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.528072+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:23:01.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.528293+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:23:01.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.528566+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:23:01.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.528825+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T08:23:01.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.529029+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T08:23:01.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.420318+0000 mon.a (mon.0) 600 : cluster [DBG] mgrmap e17: y(active, since 8m), standbys: x 2024-02-10T08:23:01.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.509094+0000 mon.a (mon.0) 601 : cluster [INF] Active manager daemon y restarted 2024-02-10T08:23:01.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.509741+0000 mon.a (mon.0) 602 : cluster [INF] Activating manager daemon y 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.522305+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.522892+0000 mon.a (mon.0) 604 : cluster [DBG] mgrmap e18: y(active, starting, since 0.0132743s), standbys: x 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.524303+0000 mon.c (mon.2) 18 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.524539+0000 mon.c (mon.2) 19 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.524797+0000 mon.c (mon.2) 20 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.525877+0000 mon.c (mon.2) 21 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.526142+0000 mon.c (mon.2) 22 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T08:23:01.880 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.526402+0000 mon.c (mon.2) 23 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.526715+0000 mon.c (mon.2) 24 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.527060+0000 mon.c (mon.2) 25 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.527398+0000 mon.c (mon.2) 26 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.527788+0000 mon.c (mon.2) 27 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.528072+0000 mon.c (mon.2) 28 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.528293+0000 mon.c (mon.2) 29 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.528566+0000 mon.c (mon.2) 30 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.528825+0000 mon.c (mon.2) 31 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T08:23:01.881 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.529029+0000 mon.c (mon.2) 32 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T08:23:01.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.529522+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T08:23:01.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: cluster 2024-02-10T08:23:00.533583+0000 mon.a (mon.0) 605 : cluster [INF] Manager daemon y is now available 2024-02-10T08:23:01.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.682865+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:01.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.685703+0000 mon.c (mon.2) 35 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:01.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.729027+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:01.882 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.763941+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:23:01.883 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.764456+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:23:01.883 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.810948+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:23:01.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[17385]: audit 2024-02-10T08:23:00.811492+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:23:01.884 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.529522+0000 mon.c (mon.2) 33 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T08:23:01.884 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: cluster 2024-02-10T08:23:00.533583+0000 mon.a (mon.0) 605 : cluster [INF] Manager daemon y is now available 2024-02-10T08:23:01.884 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.682865+0000 mon.c (mon.2) 34 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:01.884 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.685703+0000 mon.c (mon.2) 35 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:01.885 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.729027+0000 mon.a (mon.0) 606 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:01.885 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.763941+0000 mon.c (mon.2) 36 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:23:01.885 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.764456+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T08:23:01.885 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.810948+0000 mon.c (mon.2) 37 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:23:01.885 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:01 smithi067 bash[24488]: audit 2024-02-10T08:23:00.811492+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T08:23:02.417 DEBUG:teuthology.orchestra.run.smithi067:alertmanager.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@alertmanager.a.service 2024-02-10T08:23:02.420 INFO:tasks.cephadm:Adding grafana.a on smithi134 2024-02-10T08:23:02.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph orch apply grafana '1;smithi134=a' 2024-02-10T08:23:02.604 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:23:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:02 smithi134 bash[18722]: cluster 2024-02-10T08:23:01.528163+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e19: y(active, since 1.01855s), standbys: x 2024-02-10T08:23:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:02 smithi134 bash[18722]: cephadm 2024-02-10T08:23:01.539193+0000 mgr.y (mgr.14547) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi067=a;count:1 2024-02-10T08:23:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:02 smithi134 bash[18722]: audit 2024-02-10T08:23:01.543362+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:02 smithi067 bash[17385]: cluster 2024-02-10T08:23:01.528163+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e19: y(active, since 1.01855s), standbys: x 2024-02-10T08:23:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:02 smithi067 bash[17385]: cephadm 2024-02-10T08:23:01.539193+0000 mgr.y (mgr.14547) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi067=a;count:1 2024-02-10T08:23:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:02 smithi067 bash[17385]: audit 2024-02-10T08:23:01.543362+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:02 smithi067 bash[24488]: cluster 2024-02-10T08:23:01.528163+0000 mon.a (mon.0) 609 : cluster [DBG] mgrmap e19: y(active, since 1.01855s), standbys: x 2024-02-10T08:23:02.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:02 smithi067 bash[24488]: cephadm 2024-02-10T08:23:01.539193+0000 mgr.y (mgr.14547) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi067=a;count:1 2024-02-10T08:23:02.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:02 smithi067 bash[24488]: audit 2024-02-10T08:23:01.543362+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:03.699 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:03 smithi134 bash[18722]: cluster 2024-02-10T08:23:02.528045+0000 mgr.y (mgr.14547) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:03 smithi067 bash[17385]: cluster 2024-02-10T08:23:02.528045+0000 mgr.y (mgr.14547) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:03.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:03 smithi067 bash[24488]: cluster 2024-02-10T08:23:02.528045+0000 mgr.y (mgr.14547) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:04.840 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:04 smithi134 bash[18722]: cluster 2024-02-10T08:23:03.544274+0000 mon.a (mon.0) 611 : cluster [DBG] mgrmap e20: y(active, since 3s), standbys: x 2024-02-10T08:23:04.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:04 smithi067 bash[17385]: cluster 2024-02-10T08:23:03.544274+0000 mon.a (mon.0) 611 : cluster [DBG] mgrmap e20: y(active, since 3s), standbys: x 2024-02-10T08:23:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:04 smithi067 bash[24488]: cluster 2024-02-10T08:23:03.544274+0000 mon.a (mon.0) 611 : cluster [DBG] mgrmap e20: y(active, since 3s), standbys: x 2024-02-10T08:23:05.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/1237844584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f7b40 msgr2=0x7f12880f7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:05.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- 172.21.15.134:0/1237844584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f7b40 0x7f12880f7f60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f12780048a0 tx=0x7f1278009cb0).stop 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/1237844584 shutdown_connections 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- 172.21.15.134:0/1237844584 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f12880f9730 0x7f1288100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- 172.21.15.134:0/1237844584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f7b40 0x7f12880f7f60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- 172.21.15.134:0/1237844584 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 0x7f12880f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/1237844584 >> 172.21.15.134:0/1237844584 conn(0x7f12880f3340 msgr2=0x7f12880f5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/1237844584 shutdown_connections 2024-02-10T08:23:05.001 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/1237844584 wait complete. 2024-02-10T08:23:05.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 Processor -- start 2024-02-10T08:23:05.002 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- start start 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f12880f7b40 0x7f128806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 0x7f128806b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 0x7f128806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128806bbe0 con 0x7f12880f8de0 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128806bd50 con 0x7f12880f9730 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128806bf10 con 0x7f12880f7b40 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 0x7f128806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 0x7f128806b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:34124/0 (socket says 172.21.15.134:34124) 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 -- 172.21.15.134:0/401718907 learned_addr learned my addr 172.21.15.134:0/401718907 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f127effd700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 0x7f128806b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f12877fe700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f12880f7b40 0x7f128806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 -- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f12880f7b40 msgr2=0x7f128806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:05.003 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f12880f7b40 0x7f128806ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 -- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 msgr2=0x7f128806b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 0x7f128806b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 -- 172.21.15.134:0/401718907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270005040 con 0x7f12880f9730 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f127effd700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 0x7f128806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f1287fff700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 0x7f128806b5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f128000ec50 tx=0x7f128000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128000ee60 con 0x7f12880f9730 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1278005040 con 0x7f12880f9730 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f128000f040 con 0x7f12880f9730 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12881124c0 con 0x7f12880f9730 2024-02-10T08:23:05.004 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:04.998+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128000ce00 con 0x7f12880f9730 2024-02-10T08:23:05.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.002+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268004f50 con 0x7f12880f9730 2024-02-10T08:23:05.006 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.002+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f12800092e0 con 0x7f12880f9730 2024-02-10T08:23:05.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.006+0000 7f12857fa700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1274061340 0x7f1274063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:05.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.006+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f128007d1c0 con 0x7f12880f9730 2024-02-10T08:23:05.007 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.006+0000 7f12877fe700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1274061340 0x7f1274063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:05.008 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.006+0000 7f12877fe700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1274061340 0x7f1274063800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1278004ed0 tx=0x7f127800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:05.012 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.010+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f128004cf50 con 0x7f12880f9730 2024-02-10T08:23:05.206 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.202+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 --> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f1268000bc0 con 0x7f1274061340 2024-02-10T08:23:05.220 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.218+0000 7f12857fa700 1 -- 172.21.15.134:0/401718907 <== mgr.14547 v2:172.21.15.67:6800/4090043441 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f1268000bc0 con 0x7f1274061340 2024-02-10T08:23:05.221 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled grafana update... 2024-02-10T08:23:05.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1274061340 msgr2=0x7f1274063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:05.224 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1274061340 0x7f1274063800 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1278004ed0 tx=0x7f127800b040).stop 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 msgr2=0x7f128806b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 0x7f128806b5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f128000ec50 tx=0x7f128000b040).stop 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 shutdown_connections 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1274061340 0x7f1274063800 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12880f9730 0x7f128806b5c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f12880f7b40 0x7f128806ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 --2- 172.21.15.134:0/401718907 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f12880f8de0 0x7f128806b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 >> 172.21.15.134:0/401718907 conn(0x7f12880f3340 msgr2=0x7f12880f5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 shutdown_connections 2024-02-10T08:23:05.225 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:05.222+0000 7f128df85700 1 -- 172.21.15.134:0/401718907 wait complete. 2024-02-10T08:23:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:05 smithi134 bash[18722]: cluster 2024-02-10T08:23:04.528821+0000 mgr.y (mgr.14547) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:05 smithi134 bash[18722]: audit 2024-02-10T08:23:05.219526+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:05 smithi067 bash[17385]: cluster 2024-02-10T08:23:04.528821+0000 mgr.y (mgr.14547) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:05 smithi067 bash[17385]: audit 2024-02-10T08:23:05.219526+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:05.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:05 smithi067 bash[24488]: cluster 2024-02-10T08:23:04.528821+0000 mgr.y (mgr.14547) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:05 smithi067 bash[24488]: audit 2024-02-10T08:23:05.219526+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:05.971 DEBUG:teuthology.orchestra.run.smithi134:grafana.a> sudo journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@grafana.a.service 2024-02-10T08:23:05.974 INFO:tasks.cephadm:Setting up client nodes... 2024-02-10T08:23:05.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T08:23:06.012 INFO:journalctl@ceph.grafana.a.smithi134.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-02-10T08:23:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:06 smithi067 bash[24488]: audit 2024-02-10T08:23:05.208191+0000 mgr.y (mgr.14547) 7 : audit [DBG] from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:23:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:06 smithi067 bash[24488]: cephadm 2024-02-10T08:23:05.211539+0000 mgr.y (mgr.14547) 8 : cephadm [INF] Saving service grafana spec with placement smithi134=a;count:1 2024-02-10T08:23:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:06 smithi067 bash[24488]: audit 2024-02-10T08:23:06.371214+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:06 smithi067 bash[17385]: audit 2024-02-10T08:23:05.208191+0000 mgr.y (mgr.14547) 7 : audit [DBG] from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:23:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:06 smithi067 bash[17385]: cephadm 2024-02-10T08:23:05.211539+0000 mgr.y (mgr.14547) 8 : cephadm [INF] Saving service grafana spec with placement smithi134=a;count:1 2024-02-10T08:23:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:06 smithi067 bash[17385]: audit 2024-02-10T08:23:06.371214+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:06 smithi134 bash[18722]: audit 2024-02-10T08:23:05.208191+0000 mgr.y (mgr.14547) 7 : audit [DBG] from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi134=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T08:23:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:06 smithi134 bash[18722]: cephadm 2024-02-10T08:23:05.211539+0000 mgr.y (mgr.14547) 8 : cephadm [INF] Saving service grafana spec with placement smithi134=a;count:1 2024-02-10T08:23:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:06 smithi134 bash[18722]: audit 2024-02-10T08:23:06.371214+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: cluster 2024-02-10T08:23:06.529571+0000 mgr.y (mgr.14547) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:07.828 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.653136+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.653636+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.655193+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.655636+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.656879+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.657267+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.658381+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.658758+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:06.664659+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.157176+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.425997+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.426879+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.429017+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.429660+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.829 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.431419+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.431956+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.433515+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.434041+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.443708+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.445612+0000 mon.c (mon.2) 46 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.446836+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[17385]: audit 2024-02-10T08:23:07.506505+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.830 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: cluster 2024-02-10T08:23:06.529571+0000 mgr.y (mgr.14547) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.653136+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.653636+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.655193+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.655636+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.656879+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.657267+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.658381+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.658758+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:06.664659+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.157176+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.425997+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.426879+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.429017+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.831 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.429660+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.431419+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.431956+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.433515+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.434041+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.443708+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.445612+0000 mon.c (mon.2) 46 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.446836+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:23:07.832 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:07 smithi067 bash[24488]: audit 2024-02-10T08:23:07.506505+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: cluster 2024-02-10T08:23:06.529571+0000 mgr.y (mgr.14547) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.653136+0000 mon.c (mon.2) 38 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.653636+0000 mon.a (mon.0) 614 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.655193+0000 mon.c (mon.2) 39 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.655636+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.656879+0000 mon.c (mon.2) 40 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.657267+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.658381+0000 mon.c (mon.2) 41 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.658758+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:06.664659+0000 mon.a (mon.0) 618 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.157176+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.425997+0000 mon.c (mon.2) 42 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.426879+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.429017+0000 mon.c (mon.2) 43 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.429660+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.431419+0000 mon.c (mon.2) 44 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.431956+0000 mon.a (mon.0) 622 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.433515+0000 mon.c (mon.2) 45 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.434041+0000 mon.a (mon.0) 623 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.443708+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.445612+0000 mon.c (mon.2) 46 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:23:08.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.446836+0000 mon.c (mon.2) 47 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:23:08.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:07 smithi134 bash[18722]: audit 2024-02-10T08:23:07.506505+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.574 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:08 smithi067 systemd[1]: Started Ceph node-exporter.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:23:08.574 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[45939]: Unable to find image 'quay.io/prometheus/node-exporter:v1.3.1' locally 2024-02-10T08:23:08.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:06.659678+0000 mgr.y (mgr.14547) 10 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 3804M 2024-02-10T08:23:08.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:07.435177+0000 mgr.y (mgr.14547) 11 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 4060M 2024-02-10T08:23:08.831 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:07.449134+0000 mgr.y (mgr.14547) 12 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:07.449571+0000 mgr.y (mgr.14547) 13 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:07.472038+0000 mgr.y (mgr.14547) 14 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:07.493454+0000 mgr.y (mgr.14547) 15 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: audit 2024-02-10T08:23:07.567759+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: audit 2024-02-10T08:23:07.574865+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: cephadm 2024-02-10T08:23:07.582898+0000 mgr.y (mgr.14547) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi067 2024-02-10T08:23:08.832 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:08 smithi134 bash[18722]: audit 2024-02-10T08:23:08.359235+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:06.659678+0000 mgr.y (mgr.14547) 10 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 3804M 2024-02-10T08:23:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:07.435177+0000 mgr.y (mgr.14547) 11 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 4060M 2024-02-10T08:23:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:07.449134+0000 mgr.y (mgr.14547) 12 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:23:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:07.449571+0000 mgr.y (mgr.14547) 13 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:23:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:07.472038+0000 mgr.y (mgr.14547) 14 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:07.493454+0000 mgr.y (mgr.14547) 15 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: audit 2024-02-10T08:23:07.567759+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: audit 2024-02-10T08:23:07.574865+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: cephadm 2024-02-10T08:23:07.582898+0000 mgr.y (mgr.14547) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi067 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[17385]: audit 2024-02-10T08:23:08.359235+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:06.659678+0000 mgr.y (mgr.14547) 10 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 3804M 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:07.435177+0000 mgr.y (mgr.14547) 11 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 4060M 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:07.449134+0000 mgr.y (mgr.14547) 12 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.conf 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:07.449571+0000 mgr.y (mgr.14547) 13 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.conf 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:07.472038+0000 mgr.y (mgr.14547) 14 : cephadm [INF] Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:07.493454+0000 mgr.y (mgr.14547) 15 : cephadm [INF] Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-10T08:23:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: audit 2024-02-10T08:23:07.567759+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: audit 2024-02-10T08:23:07.574865+0000 mon.a (mon.0) 627 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:08.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: cephadm 2024-02-10T08:23:07.582898+0000 mgr.y (mgr.14547) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi067 2024-02-10T08:23:08.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:08 smithi067 bash[24488]: audit 2024-02-10T08:23:08.359235+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:09.098 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:09 smithi134 systemd[1]: Started Ceph node-exporter.b for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:23:09.527 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:09 smithi134 bash[40856]: Unable to find image 'quay.io/prometheus/node-exporter:v1.3.1' locally 2024-02-10T08:23:09.569 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: v1.3.1: Pulling from prometheus/node-exporter 2024-02-10T08:23:09.616 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 -- 172.21.15.67:0/372292487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fdbf0 msgr2=0x7f83600fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 --2- 172.21.15.67:0/372292487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fdbf0 0x7f83600fdfd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8350004870 tx=0x7f8350009cb0).stop 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 -- 172.21.15.67:0/372292487 shutdown_connections 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 --2- 172.21.15.67:0/372292487 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fdbf0 0x7f83600fdfd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 --2- 172.21.15.67:0/372292487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 0x7f8360104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 --2- 172.21.15.67:0/372292487 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fe550 0x7f83600f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 -- 172.21.15.67:0/372292487 >> 172.21.15.67:0/372292487 conn(0x7f8360070060 msgr2=0x7f83600f4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 -- 172.21.15.67:0/372292487 shutdown_connections 2024-02-10T08:23:09.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 -- 172.21.15.67:0/372292487 wait complete. 2024-02-10T08:23:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 Processor -- start 2024-02-10T08:23:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 -- start start 2024-02-10T08:23:09.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 0x7f83600f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.611+0000 7f8364bc2700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fdbf0 0x7f83600f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 0x7f83600f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601163b0 con 0x7f83600fe550 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360116520 con 0x7f83600f6910 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83601166e0 con 0x7f83600fdbf0 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 0x7f83600f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835dd9b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fdbf0 0x7f83600f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 0x7f83600f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43160/0 (socket says 172.21.15.67:43160) 2024-02-10T08:23:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 -- 172.21.15.67:0/3843947038 learned_addr learned my addr 172.21.15.67:0/3843947038 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:23:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835e59c700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 0x7f83600f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 -- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fdbf0 msgr2=0x7f83600f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fdbf0 0x7f83600f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 -- 172.21.15.67:0/3843947038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 msgr2=0x7f83600f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 0x7f83600f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 -- 172.21.15.67:0/3843947038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350005040 con 0x7f83600fe550 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835dd9b700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fdbf0 0x7f83600f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835e59c700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 0x7f83600f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f835ed9d700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 0x7f83600f7f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f835800c960 tx=0x7f8358004d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835800a520 con 0x7f83600fe550 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f835800ab10 con 0x7f83600fe550 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83580129d0 con 0x7f83600fe550 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83601169c0 con 0x7f83600fe550 2024-02-10T08:23:09.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8360116eb0 con 0x7f83600fe550 2024-02-10T08:23:09.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8360060ba0 con 0x7f83600fe550 2024-02-10T08:23:09.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.615+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f835800a680 con 0x7f83600fe550 2024-02-10T08:23:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.619+0000 7f834f7fe700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8348061340 0x7f8348063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.619+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f835807ca80 con 0x7f83600fe550 2024-02-10T08:23:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.619+0000 7f835e59c700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8348061340 0x7f8348063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:09.623 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.619+0000 7f835e59c700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8348061340 0x7f8348063800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f83500045d0 tx=0x7f8350009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:09.630 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.623+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8358048f60 con 0x7f83600fe550 2024-02-10T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[17385]: cephadm 2024-02-10T08:23:08.360837+0000 mgr.y (mgr.14547) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi134 2024-02-10T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[17385]: cluster 2024-02-10T08:23:08.531059+0000 mgr.y (mgr.14547) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-10T08:23:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[17385]: audit 2024-02-10T08:23:09.077386+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:09.870 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: aa2a8d90b84c: Pulling fs layer 2024-02-10T08:23:09.871 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: b45d31ee2d7f: Pulling fs layer 2024-02-10T08:23:09.871 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: b5db1e299295: Pulling fs layer 2024-02-10T08:23:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[24488]: cephadm 2024-02-10T08:23:08.360837+0000 mgr.y (mgr.14547) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi134 2024-02-10T08:23:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[24488]: cluster 2024-02-10T08:23:08.531059+0000 mgr.y (mgr.14547) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-10T08:23:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[24488]: audit 2024-02-10T08:23:09.077386+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:09.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.883+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f83600f8a70 con 0x7f83600fe550 2024-02-10T08:23:09.896 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.891+0000 7f834f7fe700 1 -- 172.21.15.67:0/3843947038 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f835804c8c0 con 0x7f83600fe550 2024-02-10T08:23:09.896 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-02-10T08:23:09.896 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQBtMsdlL7oUNRAAzJfrkZ2FcNhgcXPOc5PW5A== 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8348061340 msgr2=0x7f8348063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8348061340 0x7f8348063800 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f83500045d0 tx=0x7f8350009cb0).stop 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 msgr2=0x7f83600f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 0x7f83600f7f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f835800c960 tx=0x7f8358004d60).stop 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 shutdown_connections 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f83600fe550 0x7f83600f7f00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8348061340 0x7f8348063800 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83600f6910 0x7f83600f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.899 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 --2- 172.21.15.67:0/3843947038 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f83600fdbf0 0x7f83600f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 >> 172.21.15.67:0/3843947038 conn(0x7f8360070060 msgr2=0x7f83600fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 shutdown_connections 2024-02-10T08:23:09.900 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:09.895+0000 7f8364bc2700 1 -- 172.21.15.67:0/3843947038 wait complete. 2024-02-10T08:23:10.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:09 smithi134 bash[40856]: v1.3.1: Pulling from prometheus/node-exporter 2024-02-10T08:23:10.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:09 smithi134 bash[18722]: cephadm 2024-02-10T08:23:08.360837+0000 mgr.y (mgr.14547) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi134 2024-02-10T08:23:10.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:09 smithi134 bash[18722]: cluster 2024-02-10T08:23:08.531059+0000 mgr.y (mgr.14547) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-10T08:23:10.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:09 smithi134 bash[18722]: audit 2024-02-10T08:23:09.077386+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:10.212 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: b45d31ee2d7f: Download complete 2024-02-10T08:23:10.212 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: aa2a8d90b84c: Verifying Checksum 2024-02-10T08:23:10.212 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:09 smithi067 bash[45939]: aa2a8d90b84c: Download complete 2024-02-10T08:23:10.527 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: aa2a8d90b84c: Pulling fs layer 2024-02-10T08:23:10.527 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: b45d31ee2d7f: Pulling fs layer 2024-02-10T08:23:10.527 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: b5db1e299295: Pulling fs layer 2024-02-10T08:23:10.582 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[45939]: aa2a8d90b84c: Pull complete 2024-02-10T08:23:10.582 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[45939]: b5db1e299295: Verifying Checksum 2024-02-10T08:23:10.583 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[45939]: b5db1e299295: Download complete 2024-02-10T08:23:10.861 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[18722]: cephadm 2024-02-10T08:23:09.100847+0000 mgr.y (mgr.14547) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi134 2024-02-10T08:23:10.861 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[18722]: audit 2024-02-10T08:23:09.890256+0000 mon.a (mon.0) 630 : audit [INF] from='client.? 172.21.15.67:0/3843947038' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:10.861 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[18722]: audit 2024-02-10T08:23:09.895270+0000 mon.a (mon.0) 631 : audit [INF] from='client.? 172.21.15.67:0/3843947038' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T08:23:10.861 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: b45d31ee2d7f: Download complete 2024-02-10T08:23:10.861 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: aa2a8d90b84c: Verifying Checksum 2024-02-10T08:23:10.862 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: aa2a8d90b84c: Download complete 2024-02-10T08:23:10.862 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: b5db1e299295: Verifying Checksum 2024-02-10T08:23:10.862 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: b5db1e299295: Download complete 2024-02-10T08:23:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[17385]: cephadm 2024-02-10T08:23:09.100847+0000 mgr.y (mgr.14547) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi134 2024-02-10T08:23:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[17385]: audit 2024-02-10T08:23:09.890256+0000 mon.a (mon.0) 630 : audit [INF] from='client.? 172.21.15.67:0/3843947038' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:10.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[17385]: audit 2024-02-10T08:23:09.895270+0000 mon.a (mon.0) 631 : audit [INF] from='client.? 172.21.15.67:0/3843947038' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T08:23:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[24488]: cephadm 2024-02-10T08:23:09.100847+0000 mgr.y (mgr.14547) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi134 2024-02-10T08:23:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[24488]: audit 2024-02-10T08:23:09.890256+0000 mon.a (mon.0) 630 : audit [INF] from='client.? 172.21.15.67:0/3843947038' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:10 smithi067 bash[24488]: audit 2024-02-10T08:23:09.895270+0000 mon.a (mon.0) 631 : audit [INF] from='client.? 172.21.15.67:0/3843947038' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T08:23:11.246 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:23:11.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-10T08:23:11.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-10T08:23:11.266 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T08:23:11.274 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:10 smithi134 bash[40856]: aa2a8d90b84c: Pull complete 2024-02-10T08:23:11.527 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:11 smithi134 bash[40856]: b45d31ee2d7f: Pull complete 2024-02-10T08:23:11.576 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[45939]: b45d31ee2d7f: Pull complete 2024-02-10T08:23:11.777 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:23:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:23:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:11 smithi134 bash[18722]: cluster 2024-02-10T08:23:10.531496+0000 mgr.y (mgr.14547) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-10T08:23:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:11 smithi134 bash[18722]: audit 2024-02-10T08:23:10.735118+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:11.841 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[17385]: cluster 2024-02-10T08:23:10.531496+0000 mgr.y (mgr.14547) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-10T08:23:11.841 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[17385]: audit 2024-02-10T08:23:10.735118+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:11.842 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[24488]: cluster 2024-02-10T08:23:10.531496+0000 mgr.y (mgr.14547) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-10T08:23:11.842 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[24488]: audit 2024-02-10T08:23:10.735118+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:11.842 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[45939]: b5db1e299295: Pull complete 2024-02-10T08:23:11.842 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[45939]: Digest: sha256:f2269e73124dd0f60a7d19a2ce1264d33d08a985aed0ee6b0b89d0be470592cd 2024-02-10T08:23:11.842 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:11 smithi067 bash[45939]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.3.1 2024-02-10T08:23:12.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:12 smithi067 bash[17385]: audit 2024-02-10T08:23:11.543554+0000 mgr.y (mgr.14547) 21 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:12.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:12 smithi067 bash[24488]: audit 2024-02-10T08:23:11.543554+0000 mgr.y (mgr.14547) 21 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:12.947 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:12 smithi134 bash[18722]: audit 2024-02-10T08:23:11.543554+0000 mgr.y (mgr.14547) 21 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:13.277 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:12 smithi134 bash[40856]: b5db1e299295: Pull complete 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.060Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.060Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.060Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.060Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=arp 2024-02-10T08:23:13.371 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=edac 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-10T08:23:13.372 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=os 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-10T08:23:13.373 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=stat 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=time 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=uname 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-10T08:23:13.374 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.062Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-10T08:23:13.375 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[45939]: ts=2024-02-10T08:23:13.063Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-10T08:23:13.854 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:13 smithi134 bash[18722]: cluster 2024-02-10T08:23:12.532927+0000 mgr.y (mgr.14547) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-10T08:23:13.854 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:13 smithi134 bash[40856]: Digest: sha256:f2269e73124dd0f60a7d19a2ce1264d33d08a985aed0ee6b0b89d0be470592cd 2024-02-10T08:23:13.854 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:13 smithi134 bash[40856]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.3.1 2024-02-10T08:23:13.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[17385]: cluster 2024-02-10T08:23:12.532927+0000 mgr.y (mgr.14547) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-10T08:23:13.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:13 smithi067 bash[24488]: cluster 2024-02-10T08:23:12.532927+0000 mgr.y (mgr.14547) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-10T08:23:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:15 smithi067 bash[17385]: cluster 2024-02-10T08:23:14.533668+0000 mgr.y (mgr.14547) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:15.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:15 smithi067 bash[24488]: cluster 2024-02-10T08:23:14.533668+0000 mgr.y (mgr.14547) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[18722]: cluster 2024-02-10T08:23:14.533668+0000 mgr.y (mgr.14547) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:16.027 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.797Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-10T08:23:16.027 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.797Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-10T08:23:16.027 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.797Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.797Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.798Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=arp 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=edac 2024-02-10T08:23:16.028 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-10T08:23:16.029 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=os 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=stat 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=time 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-10T08:23:16.030 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=uname 2024-02-10T08:23:16.031 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-10T08:23:16.031 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-10T08:23:16.031 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-10T08:23:16.031 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-10T08:23:16.031 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:23:15 smithi134 bash[40856]: ts=2024-02-10T08:23:15.799Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-10T08:23:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 -- 172.21.15.134:0/3156792231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f8c50 msgr2=0x7f90dc0f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:17.789 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 --2- 172.21.15.134:0/3156792231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f8c50 0x7f90dc0f9030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f90cc004870 tx=0x7f90cc009cb0).stop 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 -- 172.21.15.134:0/3156792231 shutdown_connections 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 --2- 172.21.15.134:0/3156792231 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90dc1021b0 0x7f90dc1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 --2- 172.21.15.134:0/3156792231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f8c50 0x7f90dc0f9030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 --2- 172.21.15.134:0/3156792231 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f90dc0f95b0 0x7f90dc101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 -- 172.21.15.134:0/3156792231 >> 172.21.15.134:0/3156792231 conn(0x7f90dc0f2740 msgr2=0x7f90dc0f4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 -- 172.21.15.134:0/3156792231 shutdown_connections 2024-02-10T08:23:17.790 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 -- 172.21.15.134:0/3156792231 wait complete. 2024-02-10T08:23:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 Processor -- start 2024-02-10T08:23:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 -- start start 2024-02-10T08:23:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.786+0000 7f90e1399700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90dc0f8c50 0x7f90dc10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 0x7f90dc10e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:17.791 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f90dc1021b0 0x7f90dc108100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc1085f0 con 0x7f90dc0f8c50 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc108760 con 0x7f90dc0f95b0 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc108920 con 0x7f90dc1021b0 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 0x7f90dc10e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90daffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90dc0f8c50 0x7f90dc10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 0x7f90dc10e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:33530/0 (socket says 172.21.15.134:33530) 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 -- 172.21.15.134:0/3260514778 learned_addr learned my addr 172.21.15.134:0/3260514778 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-02-10T08:23:17.792 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90db7fe700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f90dc1021b0 0x7f90dc108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 -- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f90dc1021b0 msgr2=0x7f90dc108100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f90dc1021b0 0x7f90dc108100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 -- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90dc0f8c50 msgr2=0x7f90dc10dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90dc0f8c50 0x7f90dc10dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 -- 172.21.15.134:0/3260514778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc005040 con 0x7f90dc0f95b0 2024-02-10T08:23:17.793 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90da7fc700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 0x7f90dc10e3b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f90d000c7c0 tx=0x7f90d0007ef0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:17.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0004430 con 0x7f90dc0f95b0 2024-02-10T08:23:17.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f90d000e930 con 0x7f90dc0f95b0 2024-02-10T08:23:17.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0013960 con 0x7f90dc0f95b0 2024-02-10T08:23:17.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90dc108ba0 con 0x7f90dc0f95b0 2024-02-10T08:23:17.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc0f6a40 con 0x7f90dc0f95b0 2024-02-10T08:23:17.794 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.790+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90dc05f820 con 0x7f90dc0f95b0 2024-02-10T08:23:17.795 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.794+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f90d000eaa0 con 0x7f90dc0f95b0 2024-02-10T08:23:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.794+0000 7f90c3fff700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f90c4061340 0x7f90c4063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.794+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f90d007ca50 con 0x7f90dc0f95b0 2024-02-10T08:23:17.796 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.794+0000 7f90daffd700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f90c4061340 0x7f90c4063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:17.797 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.794+0000 7f90daffd700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f90c4061340 0x7f90c4063800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f90cc009f20 tx=0x7f90cc009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:17.802 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:17.798+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f90d004c8c0 con 0x7f90dc0f95b0 2024-02-10T08:23:17.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:17 smithi067 bash[24488]: cluster 2024-02-10T08:23:16.534827+0000 mgr.y (mgr.14547) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:17.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:17 smithi067 bash[17385]: cluster 2024-02-10T08:23:16.534827+0000 mgr.y (mgr.14547) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:17 smithi134 bash[18722]: cluster 2024-02-10T08:23:16.534827+0000 mgr.y (mgr.14547) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:18.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.074+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f90dc106f00 con 0x7f90dc0f95b0 2024-02-10T08:23:18.092 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90c3fff700 1 -- 172.21.15.134:0/3260514778 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (secure 0 0 0) 0x7f90d0018020 con 0x7f90dc0f95b0 2024-02-10T08:23:18.092 INFO:teuthology.orchestra.run.smithi134.stdout:[client.1] 2024-02-10T08:23:18.092 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQB2MsdlFHTKBBAAz9JeW4CsrGxxKpkPy1RKKg== 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f90c4061340 msgr2=0x7f90c4063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f90c4061340 0x7f90c4063800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f90cc009f20 tx=0x7f90cc009cb0).stop 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 msgr2=0x7f90dc10e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 0x7f90dc10e3b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f90d000c7c0 tx=0x7f90d0007ef0).stop 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 shutdown_connections 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f90dc0f8c50 0x7f90dc10dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f90c4061340 0x7f90c4063800 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f90dc1021b0 0x7f90dc108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 --2- 172.21.15.134:0/3260514778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90dc0f95b0 0x7f90dc10e3b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 >> 172.21.15.134:0/3260514778 conn(0x7f90dc0f2740 msgr2=0x7f90dc0f33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.090+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 shutdown_connections 2024-02-10T08:23:18.095 INFO:teuthology.orchestra.run.smithi134.stderr:2024-02-10T08:23:18.094+0000 7f90e1399700 1 -- 172.21.15.134:0/3260514778 wait complete. 2024-02-10T08:23:18.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:18 smithi134 bash[18722]: audit 2024-02-10T08:23:18.079040+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.134:0/3260514778' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:18.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:18 smithi134 bash[18722]: audit 2024-02-10T08:23:18.080042+0000 mon.a (mon.0) 633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:18.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:18 smithi134 bash[18722]: audit 2024-02-10T08:23:18.091550+0000 mon.a (mon.0) 634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T08:23:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:18 smithi067 bash[17385]: audit 2024-02-10T08:23:18.079040+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.134:0/3260514778' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:18 smithi067 bash[17385]: audit 2024-02-10T08:23:18.080042+0000 mon.a (mon.0) 633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:18 smithi067 bash[17385]: audit 2024-02-10T08:23:18.091550+0000 mon.a (mon.0) 634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T08:23:18.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:18 smithi067 bash[24488]: audit 2024-02-10T08:23:18.079040+0000 mon.b (mon.1) 26 : audit [INF] from='client.? 172.21.15.134:0/3260514778' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:18.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:18 smithi067 bash[24488]: audit 2024-02-10T08:23:18.080042+0000 mon.a (mon.0) 633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T08:23:18.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:18 smithi067 bash[24488]: audit 2024-02-10T08:23:18.091550+0000 mon.a (mon.0) 634 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T08:23:19.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:19 smithi067 bash[17385]: cluster 2024-02-10T08:23:18.536102+0000 mgr.y (mgr.14547) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:19.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:19 smithi067 bash[24488]: cluster 2024-02-10T08:23:18.536102+0000 mgr.y (mgr.14547) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:19.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:19 smithi134 bash[18722]: cluster 2024-02-10T08:23:18.536102+0000 mgr.y (mgr.14547) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:19.886 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-02-10T08:23:19.886 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-10T08:23:19.886 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-10T08:23:19.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T08:23:21.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:21 smithi067 bash[17385]: cluster 2024-02-10T08:23:20.536982+0000 mgr.y (mgr.14547) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:21.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:21 smithi067 bash[24488]: cluster 2024-02-10T08:23:20.536982+0000 mgr.y (mgr.14547) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:22.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:23:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:23:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:21 smithi134 bash[18722]: cluster 2024-02-10T08:23:20.536982+0000 mgr.y (mgr.14547) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:22.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:22 smithi067 bash[17385]: audit 2024-02-10T08:23:21.553895+0000 mgr.y (mgr.14547) 27 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:22.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:22 smithi067 bash[24488]: audit 2024-02-10T08:23:21.553895+0000 mgr.y (mgr.14547) 27 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:22 smithi134 bash[18722]: audit 2024-02-10T08:23:21.553895+0000 mgr.y (mgr.14547) 27 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:23 smithi067 bash[17385]: cluster 2024-02-10T08:23:22.538673+0000 mgr.y (mgr.14547) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:23.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:23 smithi067 bash[24488]: cluster 2024-02-10T08:23:22.538673+0000 mgr.y (mgr.14547) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:23.966 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:23 smithi134 bash[18722]: cluster 2024-02-10T08:23:22.538673+0000 mgr.y (mgr.14547) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:24.733 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:23:25.527 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 systemd[1]: Started Ceph prometheus.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:23:25.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:25 smithi067 bash[24488]: cluster 2024-02-10T08:23:24.539404+0000 mgr.y (mgr.14547) 29 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:25.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:25 smithi067 bash[24488]: audit 2024-02-10T08:23:25.144610+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:25 smithi067 bash[17385]: cluster 2024-02-10T08:23:24.539404+0000 mgr.y (mgr.14547) 29 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:25 smithi067 bash[17385]: audit 2024-02-10T08:23:25.144610+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:25.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[18722]: cluster 2024-02-10T08:23:24.539404+0000 mgr.y (mgr.14547) 29 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:25.885 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[18722]: audit 2024-02-10T08:23:25.144610+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:26.277 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.984Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.984Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.984Z caller=main.go:518 level=info host_details="(Linux 5.15.0-94-generic #104~20.04.1-Ubuntu SMP Tue Jan 16 13:34:09 UTC 2024 x86_64 smithi134 (none))" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.984Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.984Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.986Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.986Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.988Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.992Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.992Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.817µs 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.992Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T08:23:26.278 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.993Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-10T08:23:26.279 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.993Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=53.991µs wal_replay_duration=414.204µs total_replay_duration=493.529µs 2024-02-10T08:23:26.279 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.995Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T08:23:26.279 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.995Z caller=main.go:947 level=info msg="TSDB started" 2024-02-10T08:23:26.279 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:25 smithi134 bash[41423]: ts=2024-02-10T08:23:25.995Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T08:23:26.279 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:26 smithi134 bash[41423]: ts=2024-02-10T08:23:26.018Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=23.598828ms db_storage=991ns remote_storage=1.683µs web_handler=572ns query_engine=746ns scrape=424.632µs scrape_sd=53.647µs notify=1.298µs notify_sd=2.657µs rules=22.721045ms 2024-02-10T08:23:26.279 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:23:26 smithi134 bash[41423]: ts=2024-02-10T08:23:26.018Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-10T08:23:26.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:26 smithi067 bash[17385]: cephadm 2024-02-10T08:23:25.157634+0000 mgr.y (mgr.14547) 30 : cephadm [INF] Deploying daemon alertmanager.a on smithi067 2024-02-10T08:23:26.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:26 smithi067 bash[17385]: audit 2024-02-10T08:23:25.754663+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:26.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:26 smithi067 bash[24488]: cephadm 2024-02-10T08:23:25.157634+0000 mgr.y (mgr.14547) 30 : cephadm [INF] Deploying daemon alertmanager.a on smithi067 2024-02-10T08:23:26.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:26 smithi067 bash[24488]: audit 2024-02-10T08:23:25.754663+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:26 smithi134 bash[18722]: cephadm 2024-02-10T08:23:25.157634+0000 mgr.y (mgr.14547) 30 : cephadm [INF] Deploying daemon alertmanager.a on smithi067 2024-02-10T08:23:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:26 smithi134 bash[18722]: audit 2024-02-10T08:23:25.754663+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:28.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:27 smithi134 bash[18722]: cluster 2024-02-10T08:23:26.540144+0000 mgr.y (mgr.14547) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:28.036 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:27 smithi067 bash[17385]: cluster 2024-02-10T08:23:26.540144+0000 mgr.y (mgr.14547) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:28.036 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:27 smithi067 bash[24488]: cluster 2024-02-10T08:23:26.540144+0000 mgr.y (mgr.14547) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 -- 172.21.15.67:0/4024896032 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0fc2e0 msgr2=0x7f61ac0fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/4024896032 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0fc2e0 0x7f61ac0fc6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f61a0004970 tx=0x7f61a000adc0).stop 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 -- 172.21.15.67:0/4024896032 shutdown_connections 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/4024896032 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0f6b50 0x7f61ac103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/4024896032 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0fc2e0 0x7f61ac0fc6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/4024896032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 -- 172.21.15.67:0/4024896032 >> 172.21.15.67:0/4024896032 conn(0x7f61ac0701a0 msgr2=0x7f61ac0f44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:28.246 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 -- 172.21.15.67:0/4024896032 shutdown_connections 2024-02-10T08:23:28.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.239+0000 7f61b2dc2700 1 -- 172.21.15.67:0/4024896032 wait complete. 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 Processor -- start 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- start start 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0f6b50 0x7f61ac0f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fc2e0 0x7f61ac0f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac1163f0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac116560 con 0x7f61ac0fc2e0 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac116720 con 0x7f61ac0f6b50 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:28.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:46458/0 (socket says 172.21.15.67:46458) 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b0b5e700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0f6b50 0x7f61ac0f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 -- 172.21.15.67:0/2120670650 learned_addr learned my addr 172.21.15.67:0/2120670650 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61abfff700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fc2e0 0x7f61ac0f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 -- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0f6b50 msgr2=0x7f61ac0f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0f6b50 0x7f61ac0f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 -- 172.21.15.67:0/2120670650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fc2e0 msgr2=0x7f61ac0f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fc2e0 0x7f61ac0f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 -- 172.21.15.67:0/2120670650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0005040 con 0x7f61ac0fcc40 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61abfff700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fc2e0 0x7f61ac0f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-10T08:23:28.249 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b135f700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f8ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f61a400ec20 tx=0x7f61a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4009950 con 0x7f61ac0fcc40 2024-02-10T08:23:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f61a400f040 con 0x7f61ac0fcc40 2024-02-10T08:23:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac1169a0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4007e30 con 0x7f61ac0fcc40 2024-02-10T08:23:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac116ee0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.250 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.243+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac05fad0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.247+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f61a40092e0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.247+0000 7f61a9ffb700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f619c061340 0x7f619c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.247+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f61a407cbe0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.251 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.247+0000 7f61b0b5e700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f619c061340 0x7f619c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:28.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.247+0000 7f61b0b5e700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f619c061340 0x7f619c063800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f61a00043d0 tx=0x7f61a0017040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:28.257 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.251+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f61a4049f60 con 0x7f61ac0fcc40 2024-02-10T08:23:28.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.451+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61ac0f95c0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.451+0000 7f61a9ffb700 1 -- 172.21.15.67:0/2120670650 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+21155 (secure 0 0 0) 0x7f61a404cba0 con 0x7f61ac0fcc40 2024-02-10T08:23:28.458 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:23:28.458 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":61,"fsid":"2dd26d0c-c7ec-11ee-95b8-87774f69a715","created":"2024-02-10T08:13:45.488251+0000","modified":"2024-02-10T08:23:00.509187+0000","last_up_change":"2024-02-10T08:22:20.247344+0000","last_in_change":"2024-02-10T08:21:54.660923+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-10T08:18:49.528063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-10T08:22:33.283700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-10T08:22:34.781350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T08:22:36.337577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T08:22:38.307932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T08:22:40.423655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"437dcceb-1970-491f-b22e-be8d9b063290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6803","nonce":2848595980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6805","nonce":2848595980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6809","nonce":2848595980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6807","nonce":2848595980}]},"public_addr":"172.21.15.67:6803/2848595980","cluster_addr":"172.21.15.67:6805/2848595980","heartbeat_back_addr":"172.21.15.67:6809/2848595980","heartbeat_front_addr":"172.21.15.67:6807/2848595980","state":["exists","up"]},{"osd":1,"uuid":"5233196f-2e05-44ce-a209-d6266a187402","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6811","nonce":1520735968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6813","nonce":1520735968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6817","nonce":1520735968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6815","nonce":1520735968}]},"public_addr":"172.21.15.67:6811/1520735968","cluster_addr":"172.21.15.67:6813/1520735968","heartbeat_back_addr":"172.21.15.67:6817/1520735968","heartbeat_front_addr":"172.21.15.67:6815/1520735968","state":["exists","up"]},{"osd":2,"uuid":"4b812fb6-4df3-47ad-a32d-5d84545750b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6819","nonce":1580907765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6821","nonce":1580907765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6825","nonce":1580907765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6823","nonce":1580907765}]},"public_addr":"172.21.15.67:6819/1580907765","cluster_addr":"172.21.15.67:6821/1580907765","heartbeat_back_addr":"172.21.15.67:6825/1580907765","heartbeat_front_addr":"172.21.15.67:6823/1580907765","state":["exists","up"]},{"osd":3,"uuid":"13ddc51e-07b3-4e88-a568-eb7ec0dedbad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6827","nonce":2711595830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6829","nonce":2711595830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6833","nonce":2711595830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6831","nonce":2711595830}]},"public_addr":"172.21.15.67:6827/2711595830","cluster_addr":"172.21.15.67:6829/2711595830","heartbeat_back_addr":"172.21.15.67:6833/2711595830","heartbeat_front_addr":"172.21.15.67:6831/2711595830","state":["exists","up"]},{"osd":4,"uuid":"c5a34154-4009-4d77-a6f1-7b1c6a7cdd12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6801","nonce":636669687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6803","nonce":636669687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6807","nonce":636669687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6805","nonce":636669687}]},"public_addr":"172.21.15.134:6801/636669687","cluster_addr":"172.21.15.134:6803/636669687","heartbeat_back_addr":"172.21.15.134:6807/636669687","heartbeat_front_addr":"172.21.15.134:6805/636669687","state":["exists","up"]},{"osd":5,"uuid":"5f79bc97-f76e-45d8-95e9-06654d75441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6809","nonce":2103042762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6811","nonce":2103042762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6815","nonce":2103042762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6813","nonce":2103042762}]},"public_addr":"172.21.15.134:6809/2103042762","cluster_addr":"172.21.15.134:6811/2103042762","heartbeat_back_addr":"172.21.15.134:6815/2103042762","heartbeat_front_addr":"172.21.15.134:6813/2103042762","state":["exists","up"]},{"osd":6,"uuid":"92ea5976-343a-4e4c-8b06-79ee579d4a84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6817","nonce":3982264053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6819","nonce":3982264053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6823","nonce":3982264053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6821","nonce":3982264053}]},"public_addr":"172.21.15.134:6817/3982264053","cluster_addr":"172.21.15.134:6819/3982264053","heartbeat_back_addr":"172.21.15.134:6823/3982264053","heartbeat_front_addr":"172.21.15.134:6821/3982264053","state":["exists","up"]},{"osd":7,"uuid":"4ebe0339-e352-47e1-adba-42e1cd076b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6825","nonce":1760805884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6827","nonce":1760805884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6831","nonce":1760805884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6829","nonce":1760805884}]},"public_addr":"172.21.15.134:6825/1760805884","cluster_addr":"172.21.15.134:6827/1760805884","heartbeat_back_addr":"172.21.15.134:6831/1760805884","heartbeat_front_addr":"172.21.15.134:6829/1760805884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:17:19.929877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:18:03.675497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:18:47.802905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:19:32.110706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:20:12.986423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:20:54.618613+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:21:36.412286+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:22:18.451049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/4022632212":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/502171530":"2024-02-11T08:23:00.509127+0000","172.21.15.67:6801/4022632212":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/430077928":"2024-02-11T08:14:44.793111+0000","172.21.15.67:6801/2933691832":"2024-02-11T08:14:44.793111+0000","172.21.15.67:6800/2933691832":"2024-02-11T08:14:44.793111+0000","172.21.15.67:0/3228839582":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/2063401242":"2024-02-11T08:14:14.841252+0000","172.21.15.67:0/1185089911":"2024-02-11T08:14:44.793111+0000","172.21.15.67:0/1039332145":"2024-02-11T08:14:14.841252+0000","172.21.15.67:6800/2812373217":"2024-02-11T08:14:14.841252+0000","172.21.15.67:0/2508290626":"2024-02-11T08:23:00.509127+0000","172.21.15.67:6801/2812373217":"2024-02-11T08:14:14.841252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T08:23:28.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f619c061340 msgr2=0x7f619c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:28.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f619c061340 0x7f619c063800 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f61a00043d0 tx=0x7f61a0017040).stop 2024-02-10T08:23:28.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 msgr2=0x7f61ac0f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:28.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f8ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f61a400ec20 tx=0x7f61a400b040).stop 2024-02-10T08:23:28.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 shutdown_connections 2024-02-10T08:23:28.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f61ac0fcc40 0x7f61ac0f8ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f619c061340 0x7f619c063800 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f61ac0f6b50 0x7f61ac0f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 --2- 172.21.15.67:0/2120670650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61ac0fc2e0 0x7f61ac0f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:28.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 >> 172.21.15.67:0/2120670650 conn(0x7f61ac0701a0 msgr2=0x7f61ac06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:28.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 shutdown_connections 2024-02-10T08:23:28.461 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:28.455+0000 7f61b2dc2700 1 -- 172.21.15.67:0/2120670650 wait complete. 2024-02-10T08:23:28.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:28 smithi067 bash[17385]: audit 2024-02-10T08:23:28.456705+0000 mon.a (mon.0) 637 : audit [DBG] from='client.? 172.21.15.67:0/2120670650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:23:28.783 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:28 smithi067 bash[24488]: audit 2024-02-10T08:23:28.456705+0000 mon.a (mon.0) 637 : audit [DBG] from='client.? 172.21.15.67:0/2120670650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:23:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:28 smithi134 bash[18722]: audit 2024-02-10T08:23:28.456705+0000 mon.a (mon.0) 637 : audit [DBG] from='client.? 172.21.15.67:0/2120670650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:23:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:29 smithi134 bash[18722]: cluster 2024-02-10T08:23:28.540536+0000 mgr.y (mgr.14547) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:30.093 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:29 smithi067 bash[24488]: cluster 2024-02-10T08:23:28.540536+0000 mgr.y (mgr.14547) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:30.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:29 smithi067 bash[17385]: cluster 2024-02-10T08:23:28.540536+0000 mgr.y (mgr.14547) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:30.095 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-10T08:18:49.528063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-10T08:22:33.283700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-10T08:22:34.781350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-10T08:22:36.337577+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-10T08:22:38.307932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-10T08:22:40.423655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}] 2024-02-10T08:23:30.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-10T08:23:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:31 smithi067 bash[17385]: cluster 2024-02-10T08:23:30.541212+0000 mgr.y (mgr.14547) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:31 smithi067 bash[24488]: cluster 2024-02-10T08:23:30.541212+0000 mgr.y (mgr.14547) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:31 smithi134 bash[18722]: cluster 2024-02-10T08:23:30.541212+0000 mgr.y (mgr.14547) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:32.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:23:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:23:32.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:32 smithi067 bash[17385]: audit 2024-02-10T08:23:31.565309+0000 mgr.y (mgr.14547) 34 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:32.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:32 smithi067 bash[24488]: audit 2024-02-10T08:23:31.565309+0000 mgr.y (mgr.14547) 34 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:33.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:32 smithi134 bash[18722]: audit 2024-02-10T08:23:31.565309+0000 mgr.y (mgr.14547) 34 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:23:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:23:33.966 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:33 smithi067 bash[17385]: cluster 2024-02-10T08:23:32.542647+0000 mgr.y (mgr.14547) 35 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:33.967 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:33 smithi067 bash[24488]: cluster 2024-02-10T08:23:32.542647+0000 mgr.y (mgr.14547) 35 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:34.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:33 smithi134 bash[18722]: cluster 2024-02-10T08:23:32.542647+0000 mgr.y (mgr.14547) 35 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:34.518 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:34 smithi067 systemd[1]: Started Ceph alertmanager.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:23:34.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:34 smithi067 bash[17647]: [10/Feb/2024:08:23:34] ENGINE Bus STOPPING 2024-02-10T08:23:34.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:34 smithi067 bash[17647]: [10/Feb/2024:08:23:34] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:23:34.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:34 smithi067 bash[17647]: [10/Feb/2024:08:23:34] ENGINE Bus STOPPED 2024-02-10T08:23:34.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:34 smithi067 bash[17647]: [10/Feb/2024:08:23:34] ENGINE Bus STARTING 2024-02-10T08:23:34.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:34 smithi067 bash[17647]: [10/Feb/2024:08:23:34] ENGINE Serving on http://:::9283 2024-02-10T08:23:34.787 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:34 smithi067 bash[17647]: [10/Feb/2024:08:23:34] ENGINE Bus STARTED 2024-02-10T08:23:34.850 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:23:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:35 smithi134 bash[18722]: audit 2024-02-10T08:23:34.491510+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:35 smithi134 bash[18722]: cluster 2024-02-10T08:23:34.543121+0000 mgr.y (mgr.14547) 36 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:35 smithi134 bash[18722]: audit 2024-02-10T08:23:34.640835+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:35 smithi134 bash[18722]: audit 2024-02-10T08:23:34.645557+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:35 smithi134 bash[18722]: audit 2024-02-10T08:23:34.649755+0000 mon.c (mon.2) 48 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T08:23:35.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:35 smithi134 bash[18722]: audit 2024-02-10T08:23:34.655315+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[17385]: audit 2024-02-10T08:23:34.491510+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[17385]: cluster 2024-02-10T08:23:34.543121+0000 mgr.y (mgr.14547) 36 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[17385]: audit 2024-02-10T08:23:34.640835+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[17385]: audit 2024-02-10T08:23:34.645557+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[17385]: audit 2024-02-10T08:23:34.649755+0000 mon.c (mon.2) 48 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T08:23:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[17385]: audit 2024-02-10T08:23:34.655315+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[24488]: audit 2024-02-10T08:23:34.491510+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[24488]: cluster 2024-02-10T08:23:34.543121+0000 mgr.y (mgr.14547) 36 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[24488]: audit 2024-02-10T08:23:34.640835+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[24488]: audit 2024-02-10T08:23:34.645557+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[24488]: audit 2024-02-10T08:23:34.649755+0000 mon.c (mon.2) 48 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T08:23:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:35 smithi067 bash[24488]: audit 2024-02-10T08:23:34.655315+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[17385]: audit 2024-02-10T08:23:34.650217+0000 mgr.y (mgr.14547) 37 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[17385]: cephadm 2024-02-10T08:23:34.667626+0000 mgr.y (mgr.14547) 38 : cephadm [INF] Deploying daemon grafana.a on smithi134 2024-02-10T08:23:36.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[17385]: audit 2024-02-10T08:23:35.774937+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[24488]: audit 2024-02-10T08:23:34.650217+0000 mgr.y (mgr.14547) 37 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[24488]: cephadm 2024-02-10T08:23:34.667626+0000 mgr.y (mgr.14547) 38 : cephadm [INF] Deploying daemon grafana.a on smithi134 2024-02-10T08:23:36.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[24488]: audit 2024-02-10T08:23:35.774937+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:36.621 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.424Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-10T08:23:36.621 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.424Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-10T08:23:36.621 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.428Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.67 port=9094 2024-02-10T08:23:36.621 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.429Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T08:23:36.621 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.478Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T08:23:36.622 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.478Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T08:23:36.622 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.482Z caller=main.go:518 msg=Listening address=:9093 2024-02-10T08:23:36.622 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:36 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:36.482Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-10T08:23:36.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:36 smithi134 bash[18722]: audit 2024-02-10T08:23:34.650217+0000 mgr.y (mgr.14547) 37 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T08:23:36.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:36 smithi134 bash[18722]: cephadm 2024-02-10T08:23:34.667626+0000 mgr.y (mgr.14547) 38 : cephadm [INF] Deploying daemon grafana.a on smithi134 2024-02-10T08:23:36.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:36 smithi134 bash[18722]: audit 2024-02-10T08:23:35.774937+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:37.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:37 smithi134 bash[18722]: cluster 2024-02-10T08:23:36.544105+0000 mgr.y (mgr.14547) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:37.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:37 smithi067 bash[17385]: cluster 2024-02-10T08:23:36.544105+0000 mgr.y (mgr.14547) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:37 smithi067 bash[24488]: cluster 2024-02-10T08:23:36.544105+0000 mgr.y (mgr.14547) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:23:37] "GET /metrics HTTP/1.1" 200 172115 "" "Prometheus/2.33.4" 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 -- 172.21.15.67:0/3826925075 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440fdc80 msgr2=0x7fb6440fe060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 --2- 172.21.15.67:0/3826925075 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440fdc80 0x7fb6440fe060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb640004870 tx=0x7fb640009cb0).stop 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 -- 172.21.15.67:0/3826925075 shutdown_connections 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 --2- 172.21.15.67:0/3826925075 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440f84f0 0x7fb644104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 --2- 172.21.15.67:0/3826925075 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440fdc80 0x7fb6440fe060 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 --2- 172.21.15.67:0/3826925075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6440fe5e0 0x7fb6440f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 -- 172.21.15.67:0/3826925075 >> 172.21.15.67:0/3826925075 conn(0x7fb6440f38d0 msgr2=0x7fb6440f5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 -- 172.21.15.67:0/3826925075 shutdown_connections 2024-02-10T08:23:38.418 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.411+0000 7fb64c62a700 1 -- 172.21.15.67:0/3826925075 wait complete. 2024-02-10T08:23:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 Processor -- start 2024-02-10T08:23:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 -- start start 2024-02-10T08:23:38.419 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 0x7fb64410dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6440fdc80 0x7fb64410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440fe5e0 0x7fb644108180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb644108670 con 0x7fb6440fe5e0 2024-02-10T08:23:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441087e0 con 0x7fb6440fdc80 2024-02-10T08:23:38.420 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6441089a0 con 0x7fb6440f84f0 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64a3c6700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 0x7fb64410dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440fe5e0 0x7fb644108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64a3c6700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 0x7fb64410dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:47954/0 (socket says 172.21.15.67:47954) 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64a3c6700 1 -- 172.21.15.67:0/1987655090 learned_addr learned my addr 172.21.15.67:0/1987655090 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb649bc5700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6440fdc80 0x7fb64410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 -- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 msgr2=0x7fb64410dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 0x7fb64410dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 -- 172.21.15.67:0/1987655090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6440fdc80 msgr2=0x7fb64410e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6440fdc80 0x7fb64410e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 -- 172.21.15.67:0/1987655090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640005040 con 0x7fb6440fe5e0 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64a3c6700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 0x7fb64410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64abc7700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440fe5e0 0x7fb644108180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb63c00eb70 tx=0x7fb63c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c009870 con 0x7fb6440fe5e0 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fb63c00f040 con 0x7fb6440fe5e0 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb644108c20 con 0x7fb6440fe5e0 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c007c40 con 0x7fb6440fe5e0 2024-02-10T08:23:38.422 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.415+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb644114e10 con 0x7fb6440fe5e0 2024-02-10T08:23:38.423 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.419+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb644060760 con 0x7fb6440fe5e0 2024-02-10T08:23:38.424 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.419+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb63c0092e0 con 0x7fb6440fe5e0 2024-02-10T08:23:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.419+0000 7fb63b7fe700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb630061340 0x7fb630063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.419+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7fb63c07cb70 con 0x7fb6440fe5e0 2024-02-10T08:23:38.425 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.419+0000 7fb64a3c6700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb630061340 0x7fb630063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:38.426 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.419+0000 7fb64a3c6700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb630061340 0x7fb630063800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb640005010 tx=0x7fb64000b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:38.432 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.427+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb63c049f60 con 0x7fb6440fe5e0 2024-02-10T08:23:38.617 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.611+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb644109660 con 0x7fb6440fe5e0 2024-02-10T08:23:38.618 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.611+0000 7fb63b7fe700 1 -- 172.21.15.67:0/1987655090 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v61) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb63c04c9e0 con 0x7fb6440fe5e0 2024-02-10T08:23:38.618 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-10T08:23:38.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb630061340 msgr2=0x7fb630063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:38.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb630061340 0x7fb630063800 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fb640005010 tx=0x7fb64000b040).stop 2024-02-10T08:23:38.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440fe5e0 msgr2=0x7fb644108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:38.620 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440fe5e0 0x7fb644108180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fb63c00eb70 tx=0x7fb63c00b040).stop 2024-02-10T08:23:38.621 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:38 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:38.430Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000684282s 2024-02-10T08:23:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 shutdown_connections 2024-02-10T08:23:38.621 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb6440fe5e0 0x7fb644108180 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb630061340 0x7fb630063800 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb6440f84f0 0x7fb64410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 --2- 172.21.15.67:0/1987655090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6440fdc80 0x7fb64410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 >> 172.21.15.67:0/1987655090 conn(0x7fb6440f38d0 msgr2=0x7fb6440fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 shutdown_connections 2024-02-10T08:23:38.622 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:38.615+0000 7fb64c62a700 1 -- 172.21.15.67:0/1987655090 wait complete. 2024-02-10T08:23:39.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool get iscsi pg_num 2024-02-10T08:23:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:39 smithi067 bash[17385]: cluster 2024-02-10T08:23:38.544753+0000 mgr.y (mgr.14547) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:39 smithi067 bash[17385]: audit 2024-02-10T08:23:38.617201+0000 mon.a (mon.0) 643 : audit [DBG] from='client.? 172.21.15.67:0/1987655090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-10T08:23:39.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:39 smithi067 bash[24488]: cluster 2024-02-10T08:23:38.544753+0000 mgr.y (mgr.14547) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:39 smithi067 bash[24488]: audit 2024-02-10T08:23:38.617201+0000 mon.a (mon.0) 643 : audit [DBG] from='client.? 172.21.15.67:0/1987655090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-10T08:23:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:39 smithi134 bash[18722]: cluster 2024-02-10T08:23:38.544753+0000 mgr.y (mgr.14547) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:39 smithi134 bash[18722]: audit 2024-02-10T08:23:38.617201+0000 mon.a (mon.0) 643 : audit [DBG] from='client.? 172.21.15.67:0/1987655090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-10T08:23:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:41 smithi067 bash[17385]: cluster 2024-02-10T08:23:40.545491+0000 mgr.y (mgr.14547) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:41.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:41 smithi067 bash[24488]: cluster 2024-02-10T08:23:40.545491+0000 mgr.y (mgr.14547) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:42.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:41 smithi134 bash[18722]: cluster 2024-02-10T08:23:40.545491+0000 mgr.y (mgr.14547) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:42.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:23:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:23:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:42 smithi067 bash[17385]: audit 2024-02-10T08:23:41.575668+0000 mgr.y (mgr.14547) 42 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:42 smithi067 bash[24488]: audit 2024-02-10T08:23:41.575668+0000 mgr.y (mgr.14547) 42 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:43.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:42 smithi134 bash[18722]: audit 2024-02-10T08:23:41.575668+0000 mgr.y (mgr.14547) 42 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:43.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:23:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:23:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:43 smithi067 bash[17385]: cluster 2024-02-10T08:23:42.547003+0000 mgr.y (mgr.14547) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:43 smithi067 bash[24488]: cluster 2024-02-10T08:23:42.547003+0000 mgr.y (mgr.14547) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:43 smithi134 bash[18722]: cluster 2024-02-10T08:23:42.547003+0000 mgr.y (mgr.14547) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:44.188 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:23:45.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:45 smithi067 bash[24488]: cluster 2024-02-10T08:23:44.547674+0000 mgr.y (mgr.14547) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:45 smithi067 bash[17385]: cluster 2024-02-10T08:23:44.547674+0000 mgr.y (mgr.14547) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:46.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:45 smithi134 bash[18722]: cluster 2024-02-10T08:23:44.547674+0000 mgr.y (mgr.14547) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:46.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:23:46 smithi067 bash[46651]: level=info ts=2024-02-10T08:23:46.433Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003779199s 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.123+0000 7f4317379700 1 -- 172.21.15.67:0/119863243 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 msgr2=0x7f43100f6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.123+0000 7f4317379700 1 --2- 172.21.15.67:0/119863243 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f6d00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f43000047b0 tx=0x7f4300009cb0).stop 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.123+0000 7f4317379700 1 -- 172.21.15.67:0/119863243 shutdown_connections 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.123+0000 7f4317379700 1 --2- 172.21.15.67:0/119863243 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100fcb70 0x7f43100fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.123+0000 7f4317379700 1 --2- 172.21.15.67:0/119863243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100f73e0 0x7f43101039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.123+0000 7f4317379700 1 --2- 172.21.15.67:0/119863243 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f6d00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- 172.21.15.67:0/119863243 >> 172.21.15.67:0/119863243 conn(0x7f43100f27a0 msgr2=0x7f43100f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- 172.21.15.67:0/119863243 shutdown_connections 2024-02-10T08:23:47.131 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- 172.21.15.67:0/119863243 wait complete. 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 Processor -- start 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- start start 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 0x7f43100f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100fcb70 0x7f43100f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310116470 con 0x7f43100f73e0 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43101165e0 con 0x7f43100fcb70 2024-02-10T08:23:47.133 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43101167a0 con 0x7f43100fd4d0 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315115700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 0x7f43100f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315115700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 0x7f43100f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57520/0 (socket says 172.21.15.67:57520) 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315115700 1 -- 172.21.15.67:0/3367962931 learned_addr learned my addr 172.21.15.67:0/3367962931 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4314914700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100fcb70 0x7f43100f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 -- 172.21.15.67:0/3367962931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100fcb70 msgr2=0x7f43100f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100fcb70 0x7f43100f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 -- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 msgr2=0x7f43100f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:47.134 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 0x7f43100f8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 -- 172.21.15.67:0/3367962931 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0097c0 con 0x7f43100fd4d0 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4314914700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100fcb70 0x7f43100f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315115700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 0x7f43100f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4315916700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f9350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f430800e870 tx=0x7f430800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308007b40 con 0x7f43100fd4d0 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4300005040 con 0x7f43100fd4d0 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f430800f040 con 0x7f43100fd4d0 2024-02-10T08:23:47.135 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080146d0 con 0x7f43100fd4d0 2024-02-10T08:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.127+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4310116e50 con 0x7f43100fd4d0 2024-02-10T08:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.131+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43100620e0 con 0x7f43100fd4d0 2024-02-10T08:23:47.136 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.131+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4308014b00 con 0x7f43100fd4d0 2024-02-10T08:23:47.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.131+0000 7f43067fc700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f42fc061340 0x7f42fc063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:47.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.131+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7f430807caf0 con 0x7f43100fd4d0 2024-02-10T08:23:47.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.131+0000 7f4315115700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f42fc061340 0x7f42fc063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:47.137 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.131+0000 7f4315115700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f42fc061340 0x7f42fc063800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f430c001940 tx=0x7f430c004d10).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:47.141 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.135+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4308049f60 con 0x7f43100fd4d0 2024-02-10T08:23:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.323+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"} v 0) v1 -- 0x7f43100475e0 con 0x7f43100fd4d0 2024-02-10T08:23:47.328 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.323+0000 7f43067fc700 1 -- 172.21.15.67:0/3367962931 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]=0 v61) v1 ==== 94+0+10 (secure 0 0 0) 0x7f430804c8c0 con 0x7f43100fd4d0 2024-02-10T08:23:47.329 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 3 2024-02-10T08:23:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f42fc061340 msgr2=0x7f42fc063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f42fc061340 0x7f42fc063800 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f430c001940 tx=0x7f430c004d10).stop 2024-02-10T08:23:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 msgr2=0x7f43100f9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f9350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f430800e870 tx=0x7f430800b040).stop 2024-02-10T08:23:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 shutdown_connections 2024-02-10T08:23:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43100f73e0 0x7f43100f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f42fc061340 0x7f42fc063800 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.333 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43100fd4d0 0x7f43100f9350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 --2- 172.21.15.67:0/3367962931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43100fcb70 0x7f43100f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:47.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 >> 172.21.15.67:0/3367962931 conn(0x7f43100f27a0 msgr2=0x7f43100f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:47.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 shutdown_connections 2024-02-10T08:23:47.334 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:47.327+0000 7f4317379700 1 -- 172.21.15.67:0/3367962931 wait complete. 2024-02-10T08:23:47.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:47 smithi067 bash[17385]: cluster 2024-02-10T08:23:46.548966+0000 mgr.y (mgr.14547) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:47.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:47 smithi067 bash[17385]: audit 2024-02-10T08:23:47.328343+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? 172.21.15.67:0/3367962931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-10T08:23:47.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:47 smithi067 bash[24488]: cluster 2024-02-10T08:23:46.548966+0000 mgr.y (mgr.14547) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:47.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:47 smithi067 bash[24488]: audit 2024-02-10T08:23:47.328343+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? 172.21.15.67:0/3367962931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-10T08:23:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:47 smithi134 bash[18722]: cluster 2024-02-10T08:23:46.548966+0000 mgr.y (mgr.14547) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:47 smithi134 bash[18722]: audit 2024-02-10T08:23:47.328343+0000 mon.c (mon.2) 49 : audit [DBG] from='client.? 172.21.15.67:0/3367962931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-10T08:23:48.087 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:23:47] "GET /metrics HTTP/1.1" 200 172110 "" "Prometheus/2.33.4" 2024-02-10T08:23:48.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool get .rgw.root pg_num 2024-02-10T08:23:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:49 smithi134 bash[18722]: cluster 2024-02-10T08:23:48.549679+0000 mgr.y (mgr.14547) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:49 smithi067 bash[17385]: cluster 2024-02-10T08:23:48.549679+0000 mgr.y (mgr.14547) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:49 smithi067 bash[24488]: cluster 2024-02-10T08:23:48.549679+0000 mgr.y (mgr.14547) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:52.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:51 smithi134 bash[18722]: cluster 2024-02-10T08:23:50.550412+0000 mgr.y (mgr.14547) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:52.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:23:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:23:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:51 smithi067 bash[17385]: cluster 2024-02-10T08:23:50.550412+0000 mgr.y (mgr.14547) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:51 smithi067 bash[24488]: cluster 2024-02-10T08:23:50.550412+0000 mgr.y (mgr.14547) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:52.878 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:23:52.908 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:52 smithi067 bash[17385]: audit 2024-02-10T08:23:51.587041+0000 mgr.y (mgr.14547) 48 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:52.908 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:52 smithi067 bash[24488]: audit 2024-02-10T08:23:51.587041+0000 mgr.y (mgr.14547) 48 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:52 smithi134 bash[18722]: audit 2024-02-10T08:23:51.587041+0000 mgr.y (mgr.14547) 48 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:23:53.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:23:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:23:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:23:54.006 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:53 smithi067 bash[17385]: cluster 2024-02-10T08:23:52.551997+0000 mgr.y (mgr.14547) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:54.006 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:53 smithi067 bash[24488]: cluster 2024-02-10T08:23:52.551997+0000 mgr.y (mgr.14547) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:53 smithi134 bash[18722]: cluster 2024-02-10T08:23:52.551997+0000 mgr.y (mgr.14547) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 -- 172.21.15.67:0/375429790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40f8500 msgr2=0x7faae4104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 --2- 172.21.15.67:0/375429790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40f8500 0x7faae4104b00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7faadc009be0 tx=0x7faadc004d10).stop 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 -- 172.21.15.67:0/375429790 shutdown_connections 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 --2- 172.21.15.67:0/375429790 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40f8500 0x7faae4104b00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 --2- 172.21.15.67:0/375429790 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae40fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 --2- 172.21.15.67:0/375429790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40fe5f0 0x7faae40f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 -- 172.21.15.67:0/375429790 >> 172.21.15.67:0/375429790 conn(0x7faae40f3900 msgr2=0x7faae40f5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 -- 172.21.15.67:0/375429790 shutdown_connections 2024-02-10T08:23:55.691 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.682+0000 7faae9819700 1 -- 172.21.15.67:0/375429790 wait complete. 2024-02-10T08:23:55.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 Processor -- start 2024-02-10T08:23:55.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 -- start start 2024-02-10T08:23:55.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40f8500 0x7faae410dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:55.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae410e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:55.692 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40fe5f0 0x7faae4107e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4108310 con 0x7faae40fe5f0 2024-02-10T08:23:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae4108480 con 0x7faae40f8500 2024-02-10T08:23:55.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae4108640 con 0x7faae40fdc90 2024-02-10T08:23:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae27fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae410e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae27fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae410e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:39712/0 (socket says 172.21.15.67:39712) 2024-02-10T08:23:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae27fc700 1 -- 172.21.15.67:0/3985921111 learned_addr learned my addr 172.21.15.67:0/3985921111 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:23:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae37fe700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40fe5f0 0x7faae4107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40f8500 0x7faae410dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:55.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 -- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 msgr2=0x7faae410e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae410e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 -- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40fe5f0 msgr2=0x7faae4107e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40fe5f0 0x7faae4107e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 -- 172.21.15.67:0/3985921111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc0097c0 con 0x7faae40f8500 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae37fe700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40fe5f0 0x7faae4107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae27fc700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae2ffd700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40f8500 0x7faae410dc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7faad400e8f0 tx=0x7faad400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4007b40 con 0x7faae40f8500 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae4108920 con 0x7faae40f8500 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7faad400f040 con 0x7faae40f8500 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad40146d0 con 0x7faae40f8500 2024-02-10T08:23:55.695 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.686+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae4114ab0 con 0x7faae40f8500 2024-02-10T08:23:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.690+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae405fab0 con 0x7faae40f8500 2024-02-10T08:23:55.696 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.690+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7faad4009070 con 0x7faae40f8500 2024-02-10T08:23:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.690+0000 7faacbfff700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7faacc061340 0x7faacc063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:23:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.690+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7305+0+0 (secure 0 0 0) 0x7faad407c990 con 0x7faae40f8500 2024-02-10T08:23:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.690+0000 7faae27fc700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7faacc061340 0x7faacc063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:23:55.698 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.690+0000 7faae27fc700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7faacc061340 0x7faacc063800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7faad8004fd0 tx=0x7faad800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:23:55.708 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.702+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7faad4049f60 con 0x7faae40f8500 2024-02-10T08:23:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:55 smithi067 bash[17385]: cluster 2024-02-10T08:23:54.552750+0000 mgr.y (mgr.14547) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:55.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:55 smithi067 bash[24488]: cluster 2024-02-10T08:23:54.552750+0000 mgr.y (mgr.14547) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:55.897 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:55 smithi134 bash[18722]: cluster 2024-02-10T08:23:54.552750+0000 mgr.y (mgr.14547) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:55.901 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.894+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"} v 0) v1 -- 0x7faae4109300 con 0x7faae40f8500 2024-02-10T08:23:55.902 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.894+0000 7faacbfff700 1 -- 172.21.15.67:0/3985921111 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]=0 v61) v1 ==== 98+0+11 (secure 0 0 0) 0x7faad404c800 con 0x7faae40f8500 2024-02-10T08:23:55.903 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-10T08:23:55.906 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7faacc061340 msgr2=0x7faacc063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7faacc061340 0x7faacc063800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7faad8004fd0 tx=0x7faad800b040).stop 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40f8500 msgr2=0x7faae410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40f8500 0x7faae410dc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7faad400e8f0 tx=0x7faad400b040).stop 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 shutdown_connections 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7faae40fe5f0 0x7faae4107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7faacc061340 0x7faacc063800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faae40f8500 0x7faae410dc80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 --2- 172.21.15.67:0/3985921111 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7faae40fdc90 0x7faae410e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.898+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 >> 172.21.15.67:0/3985921111 conn(0x7faae40f3900 msgr2=0x7faae40fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.902+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 shutdown_connections 2024-02-10T08:23:55.907 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:23:55.902+0000 7faae9819700 1 -- 172.21.15.67:0/3985921111 wait complete. 2024-02-10T08:23:56.675 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool get default.rgw.log pg_num 2024-02-10T08:23:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:56 smithi134 bash[18722]: audit 2024-02-10T08:23:55.902136+0000 mon.b (mon.1) 27 : audit [DBG] from='client.? 172.21.15.67:0/3985921111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-10T08:23:57.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:56 smithi067 bash[17385]: audit 2024-02-10T08:23:55.902136+0000 mon.b (mon.1) 27 : audit [DBG] from='client.? 172.21.15.67:0/3985921111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-10T08:23:57.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:56 smithi067 bash[24488]: audit 2024-02-10T08:23:55.902136+0000 mon.b (mon.1) 27 : audit [DBG] from='client.? 172.21.15.67:0/3985921111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-10T08:23:58.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:57 smithi134 bash[18722]: cluster 2024-02-10T08:23:56.554121+0000 mgr.y (mgr.14547) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:57 smithi067 bash[17385]: cluster 2024-02-10T08:23:56.554121+0000 mgr.y (mgr.14547) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:58.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:23:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:23:57] "GET /metrics HTTP/1.1" 200 172110 "" "Prometheus/2.33.4" 2024-02-10T08:23:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:57 smithi067 bash[24488]: cluster 2024-02-10T08:23:56.554121+0000 mgr.y (mgr.14547) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:23:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:58 smithi134 bash[18722]: cluster 2024-02-10T08:23:58.554852+0000 mgr.y (mgr.14547) 52 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:58.778 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:58 smithi134 systemd[1]: Started Ceph grafana.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:23:59.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:58 smithi067 bash[17385]: cluster 2024-02-10T08:23:58.554852+0000 mgr.y (mgr.14547) 52 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:59.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:58 smithi067 bash[24488]: cluster 2024-02-10T08:23:58.554852+0000 mgr.y (mgr.14547) 52 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:23:59.934 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[18722]: audit 2024-02-10T08:23:58.775966+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:23:59.934 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[18722]: audit 2024-02-10T08:23:58.779777+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:23:59.934 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-02-10T08:23:59.934 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-02-10T08:23:59.934 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-02-10T08:23:59.934 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-02-10T08:23:59.934 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="App mode production" logger=settings 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-02-10T08:23:59.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-02-10T08:23:59.936 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-02-10T08:23:59.936 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-02-10T08:24:00.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:59 smithi067 bash[24488]: audit 2024-02-10T08:23:58.775966+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:00.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:23:59 smithi067 bash[24488]: audit 2024-02-10T08:23:58.779777+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:00.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:59 smithi067 bash[17385]: audit 2024-02-10T08:23:58.775966+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:00.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:23:59 smithi067 bash[17385]: audit 2024-02-10T08:23:58.779777+0000 mon.c (mon.2) 50 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:00.185 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:23:59 smithi134 bash[41763]: t=2024-02-10T08:23:59+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-02-10T08:24:00.186 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-02-10T08:24:00.186 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-02-10T08:24:00.186 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-02-10T08:24:00.186 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-02-10T08:24:00.186 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-02-10T08:24:00.438 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2024-02-10T08:24:00.439 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2024-02-10T08:24:00.692 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-02-10T08:24:00.692 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-02-10T08:24:00.692 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-02-10T08:24:00.692 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-02-10T08:24:00.692 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-02-10T08:24:00.693 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-02-10T08:24:00.693 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2024-02-10T08:24:00.693 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: audit 2024-02-10T08:24:00.541212+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: audit 2024-02-10T08:24:00.541980+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: audit 2024-02-10T08:24:00.542262+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: audit 2024-02-10T08:24:00.542857+0000 mon.c (mon.2) 53 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: audit 2024-02-10T08:24:00.543652+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: audit 2024-02-10T08:24:00.544283+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:00.962 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[18722]: cluster 2024-02-10T08:24:00.555385+0000 mgr.y (mgr.14547) 53 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:00.962 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2024-02-10T08:24:00.962 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2024-02-10T08:24:00.963 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-10T08:24:00.963 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2024-02-10T08:24:00.963 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2024-02-10T08:24:00.963 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2024-02-10T08:24:00.963 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2024-02-10T08:24:00.963 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2024-02-10T08:24:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: audit 2024-02-10T08:24:00.541212+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-10T08:24:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: audit 2024-02-10T08:24:00.541980+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: audit 2024-02-10T08:24:00.542262+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: audit 2024-02-10T08:24:00.542857+0000 mon.c (mon.2) 53 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: audit 2024-02-10T08:24:00.543652+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: audit 2024-02-10T08:24:00.544283+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[17385]: cluster 2024-02-10T08:24:00.555385+0000 mgr.y (mgr.14547) 53 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: audit 2024-02-10T08:24:00.541212+0000 mon.c (mon.2) 51 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: audit 2024-02-10T08:24:00.541980+0000 mon.c (mon.2) 52 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: audit 2024-02-10T08:24:00.542262+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: audit 2024-02-10T08:24:00.542857+0000 mon.c (mon.2) 53 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: audit 2024-02-10T08:24:00.543652+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: audit 2024-02-10T08:24:00.544283+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]: dispatch 2024-02-10T08:24:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:00 smithi067 bash[24488]: cluster 2024-02-10T08:24:00.555385+0000 mgr.y (mgr.14547) 53 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:00 smithi134 bash[41763]: t=2024-02-10T08:24:00+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-02-10T08:24:01.230 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-02-10T08:24:01.231 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-02-10T08:24:01.454 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-02-10T08:24:01.491 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-02-10T08:24:01.492 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-02-10T08:24:01.492 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-02-10T08:24:01.492 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-02-10T08:24:01.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:24:01.760 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-02-10T08:24:01.760 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-10T08:24:01.760 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-02-10T08:24:01.760 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-02-10T08:24:01.761 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-02-10T08:24:01.761 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-02-10T08:24:01.761 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-02-10T08:24:01.799 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.788173+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-10T08:24:01.799 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.788539+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-10T08:24:01.799 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.788648+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-02-10T08:24:01.799 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: cluster 2024-02-10T08:24:00.788811+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-10T08:24:01.799 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.791110+0000 mon.c (mon.2) 54 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:24:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.788173+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-10T08:24:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.788539+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-10T08:24:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.788648+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-02-10T08:24:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: cluster 2024-02-10T08:24:00.788811+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-10T08:24:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.791110+0000 mon.c (mon.2) 54 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:24:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.792044+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:24:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.797262+0000 mon.c (mon.2) 55 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:24:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.798627+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:24:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:00.801654+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:02.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[18722]: audit 2024-02-10T08:24:01.593081+0000 mgr.y (mgr.14547) 54 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:02.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-02-10T08:24:02.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-02-10T08:24:02.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-02-10T08:24:02.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-02-10T08:24:02.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-02-10T08:24:02.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:01 smithi134 bash[41763]: t=2024-02-10T08:24:01+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-02-10T08:24:02.029 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-02-10T08:24:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.792044+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:24:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.797262+0000 mon.c (mon.2) 55 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:24:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.798627+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:24:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:00.801654+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[24488]: audit 2024-02-10T08:24:01.593081+0000 mgr.y (mgr.14547) 54 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.788173+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.788539+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.788648+0000 mon.a (mon.0) 650 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [1, 2]}]': finished 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: cluster 2024-02-10T08:24:00.788811+0000 mon.a (mon.0) 651 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.791110+0000 mon.c (mon.2) 54 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.792044+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.797262+0000 mon.c (mon.2) 55 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.798627+0000 mon.a (mon.0) 653 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:00.801654+0000 mon.a (mon.0) 654 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:01 smithi067 bash[17385]: audit 2024-02-10T08:24:01.593081+0000 mgr.y (mgr.14547) 54 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-02-10T08:24:02.307 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-02-10T08:24:02.308 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-02-10T08:24:02.308 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-02-10T08:24:02.308 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-02-10T08:24:02.575 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-10T08:24:02.576 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-10T08:24:02.854 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[18722]: audit 2024-02-10T08:24:01.791753+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-10T08:24:02.854 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[18722]: cluster 2024-02-10T08:24:01.791973+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-10T08:24:02.854 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[18722]: cluster 2024-02-10T08:24:02.556245+0000 mgr.y (mgr.14547) 55 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:24:02.854 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-02-10T08:24:02.855 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-02-10T08:24:02.855 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-02-10T08:24:02.855 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2024-02-10T08:24:02.855 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2024-02-10T08:24:02.855 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2024-02-10T08:24:02.855 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2024-02-10T08:24:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:02 smithi067 bash[17385]: audit 2024-02-10T08:24:01.791753+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-10T08:24:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:02 smithi067 bash[17385]: cluster 2024-02-10T08:24:01.791973+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-10T08:24:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:02 smithi067 bash[17385]: cluster 2024-02-10T08:24:02.556245+0000 mgr.y (mgr.14547) 55 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:24:03.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:02 smithi067 bash[24488]: audit 2024-02-10T08:24:01.791753+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-10T08:24:03.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:02 smithi067 bash[24488]: cluster 2024-02-10T08:24:01.791973+0000 mon.a (mon.0) 656 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-10T08:24:03.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:02 smithi067 bash[24488]: cluster 2024-02-10T08:24:02.556245+0000 mgr.y (mgr.14547) 55 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:24:03.139 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-02-10T08:24:03.140 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-02-10T08:24:03.140 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-02-10T08:24:03.140 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-02-10T08:24:03.140 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:02 smithi134 bash[41763]: t=2024-02-10T08:24:02+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-02-10T08:24:03.141 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-02-10T08:24:03.141 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-02-10T08:24:03.141 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-02-10T08:24:03.391 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-02-10T08:24:03.392 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-02-10T08:24:03.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-02-10T08:24:03.662 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-02-10T08:24:03.929 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[18722]: cluster 2024-02-10T08:24:03.124206+0000 mon.a (mon.0) 657 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-02-10T08:24:03.930 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-02-10T08:24:04.072 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:03 smithi067 bash[17385]: cluster 2024-02-10T08:24:03.124206+0000 mon.a (mon.0) 657 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-10T08:24:04.072 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:03 smithi067 bash[24488]: cluster 2024-02-10T08:24:03.124206+0000 mon.a (mon.0) 657 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-10T08:24:04.220 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:03 smithi134 bash[41763]: t=2024-02-10T08:24:03+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-02-10T08:24:04.220 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-02-10T08:24:04.221 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-02-10T08:24:04.221 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-02-10T08:24:04.221 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-02-10T08:24:04.221 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-02-10T08:24:04.221 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2024-02-10T08:24:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.274+0000 7f15659ac700 1 -- 172.21.15.67:0/2792510174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600f84f0 msgr2=0x7f1560104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:04.276 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.274+0000 7f15659ac700 1 --2- 172.21.15.67:0/2792510174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600f84f0 0x7f1560104af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f1558009c30 tx=0x7f1558004c50).stop 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- 172.21.15.67:0/2792510174 shutdown_connections 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 --2- 172.21.15.67:0/2792510174 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600f84f0 0x7f1560104af0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 --2- 172.21.15.67:0/2792510174 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fdc80 0x7f15600fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 --2- 172.21.15.67:0/2792510174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15600fe5e0 0x7f15600f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- 172.21.15.67:0/2792510174 >> 172.21.15.67:0/2792510174 conn(0x7f15600f38d0 msgr2=0x7f15600f5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- 172.21.15.67:0/2792510174 shutdown_connections 2024-02-10T08:24:04.280 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- 172.21.15.67:0/2792510174 wait complete. 2024-02-10T08:24:04.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 Processor -- start 2024-02-10T08:24:04.281 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- start start 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15600f84f0 0x7f156010ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 0x7f156010e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 0x7f1560107f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560108450 con 0x7f15600fdc80 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601085c0 con 0x7f15600f84f0 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f15659ac700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1560108780 con 0x7f15600fe5e0 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 0x7f156010e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 0x7f1560107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 0x7f1560107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:59486/0 (socket says 172.21.15.67:59486) 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 -- 172.21.15.67:0/2533394596 learned_addr learned my addr 172.21.15.67:0/2533394596 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:04.282 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155e7fc700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 0x7f156010e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:55632/0 (socket says 172.21.15.67:55632) 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 -- 172.21.15.67:0/2533394596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15600f84f0 msgr2=0x7f156010ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15600f84f0 0x7f156010ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 -- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 msgr2=0x7f156010e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 0x7f156010e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155f7fe700 1 -- 172.21.15.67:0/2533394596 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580097c0 con 0x7f15600fe5e0 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.278+0000 7f155e7fc700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 0x7f156010e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:24:04.283 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f155f7fe700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 0x7f1560107f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f155800b870 tx=0x7f1558004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:04.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155800e040 con 0x7f15600fe5e0 2024-02-10T08:24:04.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f1558018040 con 0x7f15600fe5e0 2024-02-10T08:24:04.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1560108a00 con 0x7f15600fe5e0 2024-02-10T08:24:04.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558013d10 con 0x7f15600fe5e0 2024-02-10T08:24:04.284 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1560114bf0 con 0x7f15600fe5e0 2024-02-10T08:24:04.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f155801c600 con 0x7f15600fe5e0 2024-02-10T08:24:04.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1560060760 con 0x7f15600fe5e0 2024-02-10T08:24:04.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f1557fff700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f154c061340 0x7f154c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:04.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f155807f020 con 0x7f15600fe5e0 2024-02-10T08:24:04.286 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.282+0000 7f155effd700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f154c061340 0x7f154c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:04.287 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.286+0000 7f155effd700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f154c061340 0x7f154c063800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1548004c60 tx=0x7f154800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.290+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f15580b1f60 con 0x7f15600fe5e0 2024-02-10T08:24:04.477 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-02-10T08:24:04.477 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-02-10T08:24:04.477 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-02-10T08:24:04.478 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-02-10T08:24:04.478 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-10T08:24:04.478 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-10T08:24:04.478 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-02-10T08:24:04.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.502+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"} v 0) v1 -- 0x7f1560109440 con 0x7f15600fe5e0 2024-02-10T08:24:04.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.506+0000 7f1557fff700 1 -- 172.21.15.67:0/2533394596 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f155804dcc0 con 0x7f15600fe5e0 2024-02-10T08:24:04.508 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-10T08:24:04.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.506+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f154c061340 msgr2=0x7f154c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:04.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.506+0000 7f15659ac700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f154c061340 0x7f154c063800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1548004c60 tx=0x7f154800b040).stop 2024-02-10T08:24:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.506+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 msgr2=0x7f1560107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.506+0000 7f15659ac700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 0x7f1560107f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f155800b870 tx=0x7f1558004c50).stop 2024-02-10T08:24:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.506+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 shutdown_connections 2024-02-10T08:24:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f154c061340 0x7f154c063800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f15600fe5e0 0x7f1560107f60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15600f84f0 0x7f156010ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:24:04.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 --2- 172.21.15.67:0/2533394596 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f15600fdc80 0x7f156010e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:04.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 >> 172.21.15.67:0/2533394596 conn(0x7f15600f38d0 msgr2=0x7f15600f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:04.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 shutdown_connections 2024-02-10T08:24:04.513 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:04.510+0000 7f15659ac700 1 -- 172.21.15.67:0/2533394596 wait complete. 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-10T08:24:04.755 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:04 smithi134 bash[41763]: t=2024-02-10T08:24:04+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-02-10T08:24:05.007 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-02-10T08:24:05.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-02-10T08:24:05.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-02-10T08:24:05.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-02-10T08:24:05.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-02-10T08:24:05.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-02-10T08:24:05.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-02-10T08:24:05.528 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-02-10T08:24:05.528 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-02-10T08:24:05.528 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-02-10T08:24:05.528 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: audit 2024-02-10T08:24:04.490829+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: audit 2024-02-10T08:24:04.506912+0000 mon.c (mon.2) 56 : audit [DBG] from='client.? 172.21.15.67:0/2533394596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: cluster 2024-02-10T08:24:04.556967+0000 mgr.y (mgr.14547) 56 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: audit 2024-02-10T08:24:04.722018+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: audit 2024-02-10T08:24:04.725920+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: audit 2024-02-10T08:24:04.727082+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[18722]: audit 2024-02-10T08:24:04.733994+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool get default.rgw.control pg_num 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: audit 2024-02-10T08:24:04.490829+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: audit 2024-02-10T08:24:04.506912+0000 mon.c (mon.2) 56 : audit [DBG] from='client.? 172.21.15.67:0/2533394596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: cluster 2024-02-10T08:24:04.556967+0000 mgr.y (mgr.14547) 56 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: audit 2024-02-10T08:24:04.722018+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: audit 2024-02-10T08:24:04.725920+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: audit 2024-02-10T08:24:04.727082+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:05.754 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[24488]: audit 2024-02-10T08:24:04.733994+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: audit 2024-02-10T08:24:04.490829+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: audit 2024-02-10T08:24:04.506912+0000 mon.c (mon.2) 56 : audit [DBG] from='client.? 172.21.15.67:0/2533394596' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: cluster 2024-02-10T08:24:04.556967+0000 mgr.y (mgr.14547) 56 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.6 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: audit 2024-02-10T08:24:04.722018+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: audit 2024-02-10T08:24:04.725920+0000 mon.c (mon.2) 57 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: audit 2024-02-10T08:24:04.727082+0000 mon.c (mon.2) 58 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:05.755 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:05 smithi067 bash[17385]: audit 2024-02-10T08:24:04.733994+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:05.830 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-02-10T08:24:05.831 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:05 smithi134 bash[41763]: t=2024-02-10T08:24:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-10T08:24:06.092 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-02-10T08:24:06.352 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-02-10T08:24:06.352 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-10T08:24:06.352 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-02-10T08:24:06.352 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-02-10T08:24:06.352 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-02-10T08:24:06.352 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-02-10T08:24:06.353 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-02-10T08:24:06.353 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-02-10T08:24:06.353 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-02-10T08:24:06.606 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[18722]: cephadm 2024-02-10T08:24:04.742602+0000 mgr.y (mgr.14547) 57 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T08:24:06.607 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[18722]: cephadm 2024-02-10T08:24:04.746510+0000 mgr.y (mgr.14547) 58 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi067 2024-02-10T08:24:06.607 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-02-10T08:24:06.607 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-02-10T08:24:06.607 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-02-10T08:24:06.607 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-02-10T08:24:06.607 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-02-10T08:24:06.607 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-02-10T08:24:06.608 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-02-10T08:24:06.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:06 smithi067 bash[17385]: cephadm 2024-02-10T08:24:04.742602+0000 mgr.y (mgr.14547) 57 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T08:24:06.730 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:06 smithi067 bash[17385]: cephadm 2024-02-10T08:24:04.746510+0000 mgr.y (mgr.14547) 58 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi067 2024-02-10T08:24:06.730 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:06 smithi067 bash[24488]: cephadm 2024-02-10T08:24:04.742602+0000 mgr.y (mgr.14547) 57 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-10T08:24:06.731 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:06 smithi067 bash[24488]: cephadm 2024-02-10T08:24:04.746510+0000 mgr.y (mgr.14547) 58 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi067 2024-02-10T08:24:06.884 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-02-10T08:24:06.884 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-02-10T08:24:06.884 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-02-10T08:24:06.884 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-02-10T08:24:06.885 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-02-10T08:24:06.885 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-02-10T08:24:06.885 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:06 smithi134 bash[41763]: t=2024-02-10T08:24:06+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-02-10T08:24:07.161 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-02-10T08:24:07.162 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-02-10T08:24:07.162 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-02-10T08:24:07.162 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-02-10T08:24:07.430 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-02-10T08:24:07.430 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-02-10T08:24:07.430 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-02-10T08:24:07.431 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-02-10T08:24:07.431 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-02-10T08:24:07.431 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-02-10T08:24:07.431 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-02-10T08:24:07.431 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-02-10T08:24:07.431 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-10T08:24:07.682 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[18722]: cluster 2024-02-10T08:24:06.558159+0000 mgr.y (mgr.14547) 59 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-02-10T08:24:07.682 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-10T08:24:07.682 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-10T08:24:07.682 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-10T08:24:07.682 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-02-10T08:24:07.682 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-02-10T08:24:07.683 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-02-10T08:24:07.683 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-02-10T08:24:07.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:07 smithi067 bash[17385]: cluster 2024-02-10T08:24:06.558159+0000 mgr.y (mgr.14547) 59 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-02-10T08:24:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:07 smithi067 bash[24488]: cluster 2024-02-10T08:24:06.558159+0000 mgr.y (mgr.14547) 59 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-02-10T08:24:07.934 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-02-10T08:24:07.935 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-02-10T08:24:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:07] "GET /metrics HTTP/1.1" 200 172105 "" "Prometheus/2.33.4" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:07 smithi134 bash[41763]: t=2024-02-10T08:24:07+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-02-10T08:24:08.212 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-10T08:24:08.213 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-10T08:24:08.213 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-02-10T08:24:08.481 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-02-10T08:24:08.481 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-02-10T08:24:08.481 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-02-10T08:24:08.481 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-02-10T08:24:08.481 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-02-10T08:24:08.482 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-02-10T08:24:08.482 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-02-10T08:24:08.482 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-02-10T08:24:08.482 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-02-10T08:24:08.482 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-02-10T08:24:08.741 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-10T08:24:08.741 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-02-10T08:24:08.741 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-02-10T08:24:08.742 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-02-10T08:24:08.742 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-02-10T08:24:08.742 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-02-10T08:24:08.742 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-10T08:24:08.742 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[18722]: cluster 2024-02-10T08:24:07.498560+0000 mon.a (mon.0) 661 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-10T08:24:08.742 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[18722]: cluster 2024-02-10T08:24:07.498642+0000 mon.a (mon.0) 662 : cluster [INF] Cluster is now healthy 2024-02-10T08:24:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:08 smithi067 bash[24488]: cluster 2024-02-10T08:24:07.498560+0000 mon.a (mon.0) 661 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-10T08:24:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:08 smithi067 bash[24488]: cluster 2024-02-10T08:24:07.498642+0000 mon.a (mon.0) 662 : cluster [INF] Cluster is now healthy 2024-02-10T08:24:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:08 smithi067 bash[17385]: cluster 2024-02-10T08:24:07.498560+0000 mon.a (mon.0) 661 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-10T08:24:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:08 smithi067 bash[17385]: cluster 2024-02-10T08:24:07.498642+0000 mon.a (mon.0) 662 : cluster [INF] Cluster is now healthy 2024-02-10T08:24:08.993 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-02-10T08:24:08.994 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:08 smithi134 bash[41763]: t=2024-02-10T08:24:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-02-10T08:24:09.230 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:08 smithi067 systemd[1]: Stopping Ceph alertmanager.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:24:09.230 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:08 smithi067 bash[48071]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-alertmanager.a 2024-02-10T08:24:09.230 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:09 smithi067 bash[46651]: level=info ts=2024-02-10T08:24:09.147Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-02-10T08:24:09.254 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-02-10T08:24:09.255 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-02-10T08:24:09.506 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-02-10T08:24:09.506 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-02-10T08:24:09.507 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-02-10T08:24:09.737 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:09 smithi067 bash[17385]: cluster 2024-02-10T08:24:08.558986+0000 mgr.y (mgr.14547) 60 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-02-10T08:24:09.737 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:09 smithi067 bash[24488]: cluster 2024-02-10T08:24:08.558986+0000 mgr.y (mgr.14547) 60 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-02-10T08:24:09.758 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[18722]: cluster 2024-02-10T08:24:08.558986+0000 mgr.y (mgr.14547) 60 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 2 objects/s recovering 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-10T08:24:09.759 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-10T08:24:09.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:09 smithi067 bash[48108]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T08:24:10.019 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-02-10T08:24:10.019 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-02-10T08:24:10.020 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-10T08:24:10.020 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-10T08:24:10.020 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-02-10T08:24:10.020 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-10T08:24:10.020 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:09 smithi134 bash[41763]: t=2024-02-10T08:24:09+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-02-10T08:24:10.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-02-10T08:24:10.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-02-10T08:24:10.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-10T08:24:10.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-02-10T08:24:10.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-02-10T08:24:10.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-02-10T08:24:10.357 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:09 smithi067 bash[48173]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T08:24:10.357 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:09 smithi067 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-10T08:24:10.357 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:09 smithi067 systemd[1]: Stopped Ceph alertmanager.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:24:10.358 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:09 smithi067 systemd[1]: Started Ceph alertmanager.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:24:10.511 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:10.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-02-10T08:24:10.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-10T08:24:10.527 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-02-10T08:24:10.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-02-10T08:24:11.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[18722]: audit 2024-02-10T08:24:09.976824+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:11.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[18722]: cephadm 2024-02-10T08:24:09.979687+0000 mgr.y (mgr.14547) 61 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T08:24:11.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[18722]: cephadm 2024-02-10T08:24:09.987533+0000 mgr.y (mgr.14547) 62 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi134 2024-02-10T08:24:11.166 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[18722]: cluster 2024-02-10T08:24:10.559486+0000 mgr.y (mgr.14547) 63 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 2 objects/s recovering 2024-02-10T08:24:11.166 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:10 smithi134 bash[41763]: t=2024-02-10T08:24:10+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-02-10T08:24:11.166 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-02-10T08:24:11.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[17385]: audit 2024-02-10T08:24:09.976824+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:11.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[17385]: cephadm 2024-02-10T08:24:09.979687+0000 mgr.y (mgr.14547) 61 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T08:24:11.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[17385]: cephadm 2024-02-10T08:24:09.987533+0000 mgr.y (mgr.14547) 62 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi134 2024-02-10T08:24:11.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[17385]: cluster 2024-02-10T08:24:10.559486+0000 mgr.y (mgr.14547) 63 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 2 objects/s recovering 2024-02-10T08:24:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[24488]: audit 2024-02-10T08:24:09.976824+0000 mon.a (mon.0) 663 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[24488]: cephadm 2024-02-10T08:24:09.979687+0000 mgr.y (mgr.14547) 61 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-10T08:24:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[24488]: cephadm 2024-02-10T08:24:09.987533+0000 mgr.y (mgr.14547) 62 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi134 2024-02-10T08:24:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:10 smithi067 bash[24488]: cluster 2024-02-10T08:24:10.559486+0000 mgr.y (mgr.14547) 63 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 2 objects/s recovering 2024-02-10T08:24:11.500 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-02-10T08:24:11.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-10T08:24:11.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-02-10T08:24:11.778 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.784Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.784Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.787Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.67 port=9094 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.789Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.834Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.835Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.838Z caller=main.go:518 msg=Listening address=:9093 2024-02-10T08:24:11.983 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:11.838Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-10T08:24:12.083 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[18722]: audit 2024-02-10T08:24:11.605068+0000 mgr.y (mgr.14547) 64 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:12.083 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-02-10T08:24:12.083 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-10T08:24:12.083 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:11 smithi134 bash[41763]: t=2024-02-10T08:24:11+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-02-10T08:24:12.083 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-10T08:24:12.321 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[17385]: audit 2024-02-10T08:24:11.605068+0000 mgr.y (mgr.14547) 64 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:12.322 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:11 smithi067 bash[24488]: audit 2024-02-10T08:24:11.605068+0000 mgr.y (mgr.14547) 64 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:12.336 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-10T08:24:12.336 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-10T08:24:12.336 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-02-10T08:24:12.336 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-02-10T08:24:12.336 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-02-10T08:24:12.636 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-02-10T08:24:12.636 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-02-10T08:24:12.637 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-10T08:24:13.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-02-10T08:24:13.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-02-10T08:24:13.028 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:12 smithi134 bash[41763]: t=2024-02-10T08:24:12+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-02-10T08:24:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:13 smithi067 bash[17385]: cluster 2024-02-10T08:24:12.560861+0000 mgr.y (mgr.14547) 65 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 95 B/s wr, 4 op/s; 30 B/s, 1 objects/s recovering 2024-02-10T08:24:13.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:13 smithi067 bash[24488]: cluster 2024-02-10T08:24:12.560861+0000 mgr.y (mgr.14547) 65 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 95 B/s wr, 4 op/s; 30 B/s, 1 objects/s recovering 2024-02-10T08:24:13.443 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-02-10T08:24:13.443 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-02-10T08:24:13.443 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-02-10T08:24:13.444 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[18722]: cluster 2024-02-10T08:24:12.560861+0000 mgr.y (mgr.14547) 65 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 95 B/s wr, 4 op/s; 30 B/s, 1 objects/s recovering 2024-02-10T08:24:13.444 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:24:13.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-02-10T08:24:13.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-02-10T08:24:13.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-02-10T08:24:13.777 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-02-10T08:24:13.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 -- 172.21.15.67:0/3532381492 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0fdd10 msgr2=0x7f53ec0fe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:13.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 --2- 172.21.15.67:0/3532381492 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0fdd10 0x7f53ec0fe0f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f53dc0047b0 tx=0x7f53dc009cb0).stop 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 -- 172.21.15.67:0/3532381492 shutdown_connections 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 --2- 172.21.15.67:0/3532381492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53ec0f68d0 0x7f53ec104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 --2- 172.21.15.67:0/3532381492 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0fdd10 0x7f53ec0fe0f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 --2- 172.21.15.67:0/3532381492 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec0f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 -- 172.21.15.67:0/3532381492 >> 172.21.15.67:0/3532381492 conn(0x7f53ec0701f0 msgr2=0x7f53ec0f4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 -- 172.21.15.67:0/3532381492 shutdown_connections 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 -- 172.21.15.67:0/3532381492 wait complete. 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 Processor -- start 2024-02-10T08:24:13.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.822+0000 7f53f2041700 1 -- start start 2024-02-10T08:24:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0f68d0 0x7f53ec10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53ec0fdd10 0x7f53ec10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec108680 con 0x7f53ec0fe670 2024-02-10T08:24:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1087f0 con 0x7f53ec0fdd10 2024-02-10T08:24:13.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1089b0 con 0x7f53ec0f68d0 2024-02-10T08:24:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53eaffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53ec0fdd10 0x7f53ec10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0f68d0 0x7f53ec10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec108190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45230/0 (socket says 172.21.15.67:45230) 2024-02-10T08:24:13.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 -- 172.21.15.67:0/2387177663 learned_addr learned my addr 172.21.15.67:0/2387177663 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 -- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0f68d0 msgr2=0x7f53ec10dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0f68d0 0x7f53ec10dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 -- 172.21.15.67:0/2387177663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53ec0fdd10 msgr2=0x7f53ec10e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53ec0fdd10 0x7f53ec10e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 -- 172.21.15.67:0/2387177663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0005040 con 0x7f53ec0fe670 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53ebfff700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec108190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f53e400ec20 tx=0x7f53e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4009950 con 0x7f53ec0fe670 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f53e400f040 con 0x7f53ec0fe670 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53dc005040 con 0x7f53ec0fe670 2024-02-10T08:24:13.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4007d20 con 0x7f53ec0fe670 2024-02-10T08:24:13.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec108f90 con 0x7f53ec0fe670 2024-02-10T08:24:13.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.826+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec05fad0 con 0x7f53ec0fe670 2024-02-10T08:24:13.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.830+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f53e40092e0 con 0x7f53ec0fe670 2024-02-10T08:24:13.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.830+0000 7f53e8ff9700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f53d4061340 0x7f53d4063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.830+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f53e407cb80 con 0x7f53ec0fe670 2024-02-10T08:24:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.830+0000 7f53eb7fe700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f53d4061340 0x7f53d4063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:13.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.830+0000 7f53eb7fe700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f53d4061340 0x7f53d4063800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f53dc009bc0 tx=0x7f53dc009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:13.838 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:13.834+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f53e4049f60 con 0x7f53ec0fe670 2024-02-10T08:24:14.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.026+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"} v 0) v1 -- 0x7f53ec109670 con 0x7f53ec0fe670 2024-02-10T08:24:14.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.030+0000 7f53e8ff9700 1 -- 172.21.15.67:0/2387177663 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]=0 v63) v1 ==== 108+0+11 (secure 0 0 0) 0x7f53e404c9c0 con 0x7f53ec0fe670 2024-02-10T08:24:14.031 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f53d4061340 msgr2=0x7f53d4063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f53d4061340 0x7f53d4063800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f53dc009bc0 tx=0x7f53dc009cb0).stop 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 msgr2=0x7f53ec108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec108190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f53e400ec20 tx=0x7f53e400b040).stop 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 shutdown_connections 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f53ec0fe670 0x7f53ec108190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:14.035 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f53d4061340 0x7f53d4063800 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f53ec0f68d0 0x7f53ec10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 --2- 172.21.15.67:0/2387177663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53ec0fdd10 0x7f53ec10e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 >> 172.21.15.67:0/2387177663 conn(0x7f53ec0701f0 msgr2=0x7f53ec0fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 shutdown_connections 2024-02-10T08:24:14.036 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:14.034+0000 7f53f2041700 1 -- 172.21.15.67:0/2387177663 wait complete. 2024-02-10T08:24:14.089 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:13 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:13.790Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.001039722s 2024-02-10T08:24:14.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[18722]: audit 2024-02-10T08:24:14.030810+0000 mon.a (mon.0) 664 : audit [DBG] from='client.? 172.21.15.67:0/2387177663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-10T08:24:14.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:13 smithi134 bash[41763]: t=2024-02-10T08:24:13+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-02-10T08:24:14.278 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[41763]: t=2024-02-10T08:24:14+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-02-10T08:24:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:14 smithi067 bash[24488]: audit 2024-02-10T08:24:14.030810+0000 mon.a (mon.0) 664 : audit [DBG] from='client.? 172.21.15.67:0/2387177663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-10T08:24:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:14 smithi067 bash[17385]: audit 2024-02-10T08:24:14.030810+0000 mon.a (mon.0) 664 : audit [DBG] from='client.? 172.21.15.67:0/2387177663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-10T08:24:14.788 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[41763]: t=2024-02-10T08:24:14+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-02-10T08:24:14.788 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[41763]: t=2024-02-10T08:24:14+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-02-10T08:24:14.788 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[41763]: t=2024-02-10T08:24:14+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-02-10T08:24:14.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd pool get default.rgw.meta pg_num 2024-02-10T08:24:15.068 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[41763]: t=2024-02-10T08:24:14+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-02-10T08:24:15.068 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:14 smithi134 bash[41763]: t=2024-02-10T08:24:14+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-02-10T08:24:15.068 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-02-10T08:24:15.364 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[18722]: cluster 2024-02-10T08:24:14.561458+0000 mgr.y (mgr.14547) 66 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T08:24:15.365 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-02-10T08:24:15.365 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-02-10T08:24:15.365 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-02-10T08:24:15.365 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-02-10T08:24:15.365 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-02-10T08:24:15.365 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-02-10T08:24:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:15 smithi067 bash[17385]: cluster 2024-02-10T08:24:14.561458+0000 mgr.y (mgr.14547) 66 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T08:24:15.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:15 smithi067 bash[24488]: cluster 2024-02-10T08:24:14.561458+0000 mgr.y (mgr.14547) 66 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T08:24:15.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-02-10T08:24:15.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-02-10T08:24:15.661 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-02-10T08:24:16.027 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-02-10T08:24:16.027 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:15 smithi134 bash[41763]: t=2024-02-10T08:24:15+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-02-10T08:24:16.428 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-02-10T08:24:16.428 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-02-10T08:24:16.428 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-02-10T08:24:16.428 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-02-10T08:24:16.701 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 systemd[1]: Stopping Ceph prometheus.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:24:16.701 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[42377]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-prometheus.a 2024-02-10T08:24:16.701 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-02-10T08:24:16.701 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-02-10T08:24:16.701 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-02-10T08:24:16.701 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-02-10T08:24:16.701 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.728Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.728Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.728Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.728Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.728Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.728Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.729Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.729Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.729Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-10T08:24:16.987 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.729Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T08:24:16.988 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.730Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-10T08:24:16.988 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41423]: ts=2024-02-10T08:24:16.730Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-10T08:24:16.988 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-02-10T08:24:16.988 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-02-10T08:24:16.988 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-02-10T08:24:16.988 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-02-10T08:24:16.988 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:16 smithi134 bash[41763]: t=2024-02-10T08:24:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-02-10T08:24:17.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=17.34326966s 2024-02-10T08:24:17.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-02-10T08:24:17.322 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-02-10T08:24:17.590 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-02-10T08:24:17.590 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-02-10T08:24:17.590 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-02-10T08:24:17.590 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-02-10T08:24:17.845 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[42412]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T08:24:17.845 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[18722]: cluster 2024-02-10T08:24:16.562949+0000 mgr.y (mgr.14547) 67 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T08:24:17.846 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-02-10T08:24:17.846 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-10T08:24:17.846 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-02-10T08:24:17.846 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-02-10T08:24:17.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:17 smithi067 bash[17385]: cluster 2024-02-10T08:24:16.562949+0000 mgr.y (mgr.14547) 67 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T08:24:17.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:17 smithi067 bash[24488]: cluster 2024-02-10T08:24:16.562949+0000 mgr.y (mgr.14547) 67 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 4 op/s; 27 B/s, 1 objects/s recovering 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:17 smithi067 bash[17647]: [10/Feb/2024:08:24:17] ENGINE Bus STOPPING 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:17 smithi067 bash[17647]: [10/Feb/2024:08:24:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:17 smithi067 bash[17647]: [10/Feb/2024:08:24:17] ENGINE Bus STOPPED 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:17 smithi067 bash[17647]: [10/Feb/2024:08:24:17] ENGINE Bus STARTING 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Serving on http://:::9283 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STARTED 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STOPPING 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STOPPED 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STARTING 2024-02-10T08:24:18.270 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Serving on http://:::9283 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STARTED 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STOPPING 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STOPPED 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STARTING 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Serving on http://:::9283 2024-02-10T08:24:18.271 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17647]: [10/Feb/2024:08:24:18] ENGINE Bus STARTED 2024-02-10T08:24:18.277 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[41763]: t=2024-02-10T08:24:17+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-02-10T08:24:18.277 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:17 smithi134 bash[42478]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T08:24:18.277 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:17 smithi134 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@prometheus.a.service: Succeeded. 2024-02-10T08:24:18.277 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:17 smithi134 systemd[1]: Stopped Ceph prometheus.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:24:18.277 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:17 smithi134 systemd[1]: Started Ceph prometheus.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.890507+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.896860+0000 mon.c (mon.2) 59 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.897521+0000 mgr.y (mgr.14547) 68 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.899082+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi067.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.899657+0000 mgr.y (mgr.14547) 69 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi067.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.907873+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.924102+0000 mon.c (mon.2) 61 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.924814+0000 mgr.y (mgr.14547) 70 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T08:24:19.191 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.929254+0000 mon.c (mon.2) 62 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi134.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.929836+0000 mgr.y (mgr.14547) 71 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi134.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.939340+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.950588+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.951284+0000 mgr.y (mgr.14547) 72 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.955911+0000 mon.c (mon.2) 64 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi134.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.956497+0000 mgr.y (mgr.14547) 73 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi134.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.965547+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: audit 2024-02-10T08:24:17.973233+0000 mon.c (mon.2) 65 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:19.192 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:18 smithi134 bash[18722]: cluster 2024-02-10T08:24:18.563682+0000 mgr.y (mgr.14547) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:19.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.890507+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.896860+0000 mon.c (mon.2) 59 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T08:24:19.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.897521+0000 mgr.y (mgr.14547) 68 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T08:24:19.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.899082+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi067.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T08:24:19.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.899657+0000 mgr.y (mgr.14547) 69 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi067.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T08:24:19.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.907873+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.924102+0000 mon.c (mon.2) 61 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T08:24:19.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.924814+0000 mgr.y (mgr.14547) 70 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T08:24:19.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.929254+0000 mon.c (mon.2) 62 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi134.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T08:24:19.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.929836+0000 mgr.y (mgr.14547) 71 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi134.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T08:24:19.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.939340+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.950588+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T08:24:19.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.951284+0000 mgr.y (mgr.14547) 72 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T08:24:19.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.955911+0000 mon.c (mon.2) 64 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi134.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T08:24:19.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.956497+0000 mgr.y (mgr.14547) 73 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi134.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T08:24:19.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.965547+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: audit 2024-02-10T08:24:17.973233+0000 mon.c (mon.2) 65 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:19.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[17385]: cluster 2024-02-10T08:24:18.563682+0000 mgr.y (mgr.14547) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:19.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.890507+0000 mon.a (mon.0) 665 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.896860+0000 mon.c (mon.2) 59 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T08:24:19.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.897521+0000 mgr.y (mgr.14547) 68 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-10T08:24:19.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.899082+0000 mon.c (mon.2) 60 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi067.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T08:24:19.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.899657+0000 mgr.y (mgr.14547) 69 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi067.front.sepia.ceph.com:9093"}]: dispatch 2024-02-10T08:24:19.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.907873+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.924102+0000 mon.c (mon.2) 61 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T08:24:19.376 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.924814+0000 mgr.y (mgr.14547) 70 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-10T08:24:19.376 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.929254+0000 mon.c (mon.2) 62 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi134.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T08:24:19.376 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.929836+0000 mgr.y (mgr.14547) 71 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi134.front.sepia.ceph.com:9095"}]: dispatch 2024-02-10T08:24:19.376 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.939340+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.377 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.950588+0000 mon.c (mon.2) 63 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T08:24:19.377 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.951284+0000 mgr.y (mgr.14547) 72 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-10T08:24:19.377 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.955911+0000 mon.c (mon.2) 64 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi134.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T08:24:19.377 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.956497+0000 mgr.y (mgr.14547) 73 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi134.front.sepia.ceph.com:3000"}]: dispatch 2024-02-10T08:24:19.377 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.965547+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:19.377 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: audit 2024-02-10T08:24:17.973233+0000 mon.c (mon.2) 65 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:19.378 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:18 smithi067 bash[24488]: cluster 2024-02-10T08:24:18.563682+0000 mgr.y (mgr.14547) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:19.510 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.189Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-10T08:24:19.510 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.190Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.190Z caller=main.go:518 level=info host_details="(Linux 5.15.0-94-generic #104~20.04.1-Ubuntu SMP Tue Jan 16 13:34:09 UTC 2024 x86_64 smithi134 (none))" 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.190Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.190Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.194Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.194Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.196Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.198Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.198Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.535µs 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.198Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T08:24:19.511 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.509Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-10T08:24:19.693 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:19.777 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.510Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-10T08:24:19.778 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.510Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=41.899µs wal_replay_duration=311.850366ms total_replay_duration=311.923444ms 2024-02-10T08:24:19.778 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.514Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T08:24:19.778 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.514Z caller=main.go:947 level=info msg="TSDB started" 2024-02-10T08:24:19.778 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.514Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T08:24:19.778 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.544Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=29.268016ms db_storage=1.266µs remote_storage=2.87µs web_handler=601ns query_engine=1.145µs scrape=383.246µs scrape_sd=45.507µs notify=33.123µs notify_sd=10.383µs rules=28.236929ms 2024-02-10T08:24:19.778 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:24:19 smithi134 bash[42505]: ts=2024-02-10T08:24:19.544Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-10T08:24:21.794 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:21 smithi067 bash[17385]: cluster 2024-02-10T08:24:20.564481+0000 mgr.y (mgr.14547) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:21.795 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:21 smithi067 bash[24488]: cluster 2024-02-10T08:24:20.564481+0000 mgr.y (mgr.14547) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:21 smithi134 bash[18722]: cluster 2024-02-10T08:24:20.564481+0000 mgr.y (mgr.14547) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:22.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:24:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:24:22.120 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:24:21 smithi067 bash[48202]: level=info ts=2024-02-10T08:24:21.793Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.004382116s 2024-02-10T08:24:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- 172.21.15.67:0/3737906361 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fe5f0 msgr2=0x7f17b00f7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:22.440 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- 172.21.15.67:0/3737906361 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fe5f0 0x7f17b00f7e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f17980047b0 tx=0x7f1798009cb0).stop 2024-02-10T08:24:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- 172.21.15.67:0/3737906361 shutdown_connections 2024-02-10T08:24:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- 172.21.15.67:0/3737906361 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00f8500 0x7f17b0104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- 172.21.15.67:0/3737906361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b00fdc90 0x7f17b00fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- 172.21.15.67:0/3737906361 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fe5f0 0x7f17b00f7e20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- 172.21.15.67:0/3737906361 >> 172.21.15.67:0/3737906361 conn(0x7f17b00f38c0 msgr2=0x7f17b00f5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:22.441 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- 172.21.15.67:0/3737906361 shutdown_connections 2024-02-10T08:24:22.442 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- 172.21.15.67:0/3737906361 wait complete. 2024-02-10T08:24:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 Processor -- start 2024-02-10T08:24:22.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- start start 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b00f8500 0x7f17b010dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 0x7f17b010e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00fe5f0 0x7f17b0108170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0108660 con 0x7f17b00fe5f0 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b01087d0 con 0x7f17b00f8500 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b6f11700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0108990 con 0x7f17b00fdc90 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17affff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 0x7f17b010e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17affff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 0x7f17b010e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:55418/0 (socket says 172.21.15.67:55418) 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17affff700 1 -- 172.21.15.67:0/3660680620 learned_addr learned my addr 172.21.15.67:0/3660680620 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b4cad700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b00f8500 0x7f17b010dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.438+0000 7f17b54ae700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00fe5f0 0x7f17b0108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17affff700 1 -- 172.21.15.67:0/3660680620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b00f8500 msgr2=0x7f17b010dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17affff700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b00f8500 0x7f17b010dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17affff700 1 -- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00fe5f0 msgr2=0x7f17b0108170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17affff700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00fe5f0 0x7f17b0108170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17affff700 1 -- 172.21.15.67:0/3660680620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1798005040 con 0x7f17b00fdc90 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17b54ae700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00fe5f0 0x7f17b0108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17affff700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 0x7f17b010e4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1798000e50 tx=0x7f1798009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1798018070 con 0x7f17b00fdc90 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17b6f11700 1 -- 172.21.15.67:0/3660680620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b0108c10 con 0x7f17b00fdc90 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f17980096d0 con 0x7f17b00fdc90 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179801db90 con 0x7f17b00fdc90 2024-02-10T08:24:22.446 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17b6f11700 1 -- 172.21.15.67:0/3660680620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b0108fe0 con 0x7f17b00fdc90 2024-02-10T08:24:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1794004f20 con 0x7f17b00fdc90 2024-02-10T08:24:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1798027440 con 0x7f17b00fdc90 2024-02-10T08:24:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17adffb700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f179c061340 0x7f179c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f179807e390 con 0x7f17b00fdc90 2024-02-10T08:24:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.442+0000 7f17b4cad700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f179c061340 0x7f179c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:22.448 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.446+0000 7f17b4cad700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f179c061340 0x7f179c063800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f17a0007f30 tx=0x7f17a0009040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:22.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.450+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f179804e3a0 con 0x7f17b00fdc90 2024-02-10T08:24:22.636 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.634+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"} v 0) v1 -- 0x7f1794005750 con 0x7f17b00fdc90 2024-02-10T08:24:22.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.634+0000 7f17adffb700 1 -- 172.21.15.67:0/3660680620 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]=0 v63) v1 ==== 105+0+11 (secure 0 0 0) 0x7f17980519c0 con 0x7f17b00fdc90 2024-02-10T08:24:22.637 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 32 2024-02-10T08:24:22.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f179c061340 msgr2=0x7f179c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:22.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f179c061340 0x7f179c063800 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f17a0007f30 tx=0x7f17a0009040).stop 2024-02-10T08:24:22.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 msgr2=0x7f17b010e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:22.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 0x7f17b010e4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1798000e50 tx=0x7f1798009cb0).stop 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 shutdown_connections 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f17b00fe5f0 0x7f17b0108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f179c061340 0x7f179c063800 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17b00f8500 0x7f17b010dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 --2- 172.21.15.67:0/3660680620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f17b00fdc90 0x7f17b010e4c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 >> 172.21.15.67:0/3660680620 conn(0x7f17b00f38c0 msgr2=0x7f17b00f54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 shutdown_connections 2024-02-10T08:24:22.640 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:22.638+0000 7f17a77fe700 1 -- 172.21.15.67:0/3660680620 wait complete. 2024-02-10T08:24:22.937 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:22 smithi067 bash[17385]: audit 2024-02-10T08:24:21.616511+0000 mgr.y (mgr.14547) 76 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:22.937 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:22 smithi067 bash[24488]: audit 2024-02-10T08:24:21.616511+0000 mgr.y (mgr.14547) 76 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:22 smithi134 bash[18722]: audit 2024-02-10T08:24:21.616511+0000 mgr.y (mgr.14547) 76 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:23.538 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-10T08:24:23.538 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-10T08:24:23.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph mgr dump --format=json 2024-02-10T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:23 smithi067 bash[24488]: cluster 2024-02-10T08:24:22.565879+0000 mgr.y (mgr.14547) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:23 smithi067 bash[24488]: audit 2024-02-10T08:24:22.636819+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? 172.21.15.67:0/3660680620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-10T08:24:23.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:23 smithi067 bash[24488]: audit 2024-02-10T08:24:23.547239+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:23.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:23 smithi067 bash[17385]: cluster 2024-02-10T08:24:22.565879+0000 mgr.y (mgr.14547) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:23.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:23 smithi067 bash[17385]: audit 2024-02-10T08:24:22.636819+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? 172.21.15.67:0/3660680620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-10T08:24:23.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:23 smithi067 bash[17385]: audit 2024-02-10T08:24:23.547239+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:24.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:23 smithi134 bash[18722]: cluster 2024-02-10T08:24:22.565879+0000 mgr.y (mgr.14547) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:24.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:23 smithi134 bash[18722]: audit 2024-02-10T08:24:22.636819+0000 mon.c (mon.2) 66 : audit [DBG] from='client.? 172.21.15.67:0/3660680620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-10T08:24:24.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:23 smithi134 bash[18722]: audit 2024-02-10T08:24:23.547239+0000 mon.a (mon.0) 669 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:25 smithi134 bash[18722]: cluster 2024-02-10T08:24:24.566533+0000 mgr.y (mgr.14547) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:25 smithi134 bash[18722]: audit 2024-02-10T08:24:24.991484+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:25 smithi134 bash[18722]: audit 2024-02-10T08:24:25.260409+0000 mon.c (mon.2) 67 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:25 smithi134 bash[18722]: audit 2024-02-10T08:24:25.262261+0000 mon.c (mon.2) 68 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:26.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:25 smithi134 bash[18722]: audit 2024-02-10T08:24:25.272894+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:25 smithi134 bash[18722]: audit 2024-02-10T08:24:25.286639+0000 mon.c (mon.2) 69 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[17385]: cluster 2024-02-10T08:24:24.566533+0000 mgr.y (mgr.14547) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[17385]: audit 2024-02-10T08:24:24.991484+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[17385]: audit 2024-02-10T08:24:25.260409+0000 mon.c (mon.2) 67 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[17385]: audit 2024-02-10T08:24:25.262261+0000 mon.c (mon.2) 68 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[17385]: audit 2024-02-10T08:24:25.272894+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[17385]: audit 2024-02-10T08:24:25.286639+0000 mon.c (mon.2) 69 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[24488]: cluster 2024-02-10T08:24:24.566533+0000 mgr.y (mgr.14547) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[24488]: audit 2024-02-10T08:24:24.991484+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[24488]: audit 2024-02-10T08:24:25.260409+0000 mon.c (mon.2) 67 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[24488]: audit 2024-02-10T08:24:25.262261+0000 mon.c (mon.2) 68 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[24488]: audit 2024-02-10T08:24:25.272894+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:25 smithi067 bash[24488]: audit 2024-02-10T08:24:25.286639+0000 mon.c (mon.2) 69 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:24:28.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:27 smithi134 bash[18722]: cluster 2024-02-10T08:24:26.568000+0000 mgr.y (mgr.14547) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:27 smithi067 bash[17385]: cluster 2024-02-10T08:24:26.568000+0000 mgr.y (mgr.14547) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:27 smithi067 bash[24488]: cluster 2024-02-10T08:24:26.568000+0000 mgr.y (mgr.14547) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:28.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:27] "GET /metrics HTTP/1.1" 200 172149 "" "Prometheus/2.33.4" 2024-02-10T08:24:28.333 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:29 smithi134 bash[18722]: cluster 2024-02-10T08:24:28.568627+0000 mgr.y (mgr.14547) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:29 smithi067 bash[17385]: cluster 2024-02-10T08:24:28.568627+0000 mgr.y (mgr.14547) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:29 smithi067 bash[24488]: cluster 2024-02-10T08:24:28.568627+0000 mgr.y (mgr.14547) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- 172.21.15.67:0/4232720985 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f8e70 msgr2=0x7f84e00f9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/4232720985 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f8e70 0x7f84e00f9250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f84d00047b0 tx=0x7f84d0009cb0).stop 2024-02-10T08:24:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- 172.21.15.67:0/4232720985 shutdown_connections 2024-02-10T08:24:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/4232720985 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.180 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/4232720985 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f8e70 0x7f84e00f9250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/4232720985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e00f97d0 0x7f84e0101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- 172.21.15.67:0/4232720985 >> 172.21.15.67:0/4232720985 conn(0x7f84e00f27e0 msgr2=0x7f84e00f4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- 172.21.15.67:0/4232720985 shutdown_connections 2024-02-10T08:24:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- 172.21.15.67:0/4232720985 wait complete. 2024-02-10T08:24:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 Processor -- start 2024-02-10T08:24:31.181 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- start start 2024-02-10T08:24:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e00f8e70 0x7f84e00f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 0x7f84e00f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e00f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0116550 con 0x7f84e0102340 2024-02-10T08:24:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e01166c0 con 0x7f84e00f8e70 2024-02-10T08:24:31.182 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e6a4b700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0116880 con 0x7f84e00f97d0 2024-02-10T08:24:31.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e4fe8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e00f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:31.183 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84df7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 0x7f84e00f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e4fe8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e00f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33392/0 (socket says 172.21.15.67:33392) 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84e4fe8700 1 -- 172.21.15.67:0/3361541064 learned_addr learned my addr 172.21.15.67:0/3361541064 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84df7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 0x7f84e00f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:45438/0 (socket says 172.21.15.67:45438) 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.178+0000 7f84dffff700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e00f8e70 0x7f84e00f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e4fe8700 1 -- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 msgr2=0x7f84e00f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e4fe8700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 0x7f84e00f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e4fe8700 1 -- 172.21.15.67:0/3361541064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e00f8e70 msgr2=0x7f84e00f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e4fe8700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e00f8e70 0x7f84e00f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e4fe8700 1 -- 172.21.15.67:0/3361541064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80097c0 con 0x7f84e0102340 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84df7fe700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 0x7f84e00f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:24:31.184 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e4fe8700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e00f7640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f84d8001df0 tx=0x7f84d8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8009a10 con 0x7f84e0102340 2024-02-10T08:24:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d0005040 con 0x7f84e0102340 2024-02-10T08:24:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f84d80090d0 con 0x7f84e0102340 2024-02-10T08:24:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8007be0 con 0x7f84e0102340 2024-02-10T08:24:31.185 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e0116f40 con 0x7f84e0102340 2024-02-10T08:24:31.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc004f50 con 0x7f84e0102340 2024-02-10T08:24:31.186 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.182+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f84d80094b0 con 0x7f84e0102340 2024-02-10T08:24:31.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.186+0000 7f84dd7fa700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f84c8061340 0x7f84c8063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:31.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.186+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f84d807d750 con 0x7f84e0102340 2024-02-10T08:24:31.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.186+0000 7f84dffff700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f84c8061340 0x7f84c8063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:31.188 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.186+0000 7f84dffff700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f84c8061340 0x7f84c8063800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f84d0004e20 tx=0x7f84d0009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:31.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.194+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f84d804d430 con 0x7f84e0102340 2024-02-10T08:24:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:31 smithi067 bash[17385]: audit 2024-02-10T08:24:30.104168+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:31 smithi067 bash[17385]: cluster 2024-02-10T08:24:30.569274+0000 mgr.y (mgr.14547) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:31 smithi067 bash[24488]: audit 2024-02-10T08:24:30.104168+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:31 smithi067 bash[24488]: cluster 2024-02-10T08:24:30.569274+0000 mgr.y (mgr.14547) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:31.434 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.430+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f84cc005200 con 0x7f84e0102340 2024-02-10T08:24:31.437 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.434+0000 7f84dd7fa700 1 -- 172.21.15.67:0/3361541064 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v20) v1 ==== 74+0+151997 (secure 0 0 0) 0x7f84d8009240 con 0x7f84e0102340 2024-02-10T08:24:31.440 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f84c8061340 msgr2=0x7f84c8063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f84c8061340 0x7f84c8063800 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f84d0004e20 tx=0x7f84d0009cb0).stop 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 msgr2=0x7f84e00f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e00f7640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f84d8001df0 tx=0x7f84d8004d10).stop 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 shutdown_connections 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f84e0102340 0x7f84e00f7640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f84c8061340 0x7f84c8063800 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e00f8e70 0x7f84e00f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 --2- 172.21.15.67:0/3361541064 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f84e00f97d0 0x7f84e00f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 >> 172.21.15.67:0/3361541064 conn(0x7f84e00f27e0 msgr2=0x7f84e00f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 shutdown_connections 2024-02-10T08:24:31.444 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:31.442+0000 7f84e6a4b700 1 -- 172.21.15.67:0/3361541064 wait complete. 2024-02-10T08:24:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:31 smithi134 bash[18722]: audit 2024-02-10T08:24:30.104168+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:31 smithi134 bash[18722]: cluster 2024-02-10T08:24:30.569274+0000 mgr.y (mgr.14547) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:32.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:24:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:24:32.187 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":20,"active_gid":14547,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":4090043441},{"type":"v1","addr":"172.21.15.67:6801","nonce":4090043441}]},"active_addr":"172.21.15.67:6801/4090043441","active_change":"2024-02-10T08:23:00.509601+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":24460,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/","prometheus":"http://172.21.15.67:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":61,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2901716354}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":2308715615}]}]} 2024-02-10T08:24:32.191 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-10T08:24:32.191 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-10T08:24:32.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T08:24:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[17385]: audit 2024-02-10T08:24:31.342378+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[17385]: audit 2024-02-10T08:24:31.344459+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[17385]: audit 2024-02-10T08:24:31.345742+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[17385]: audit 2024-02-10T08:24:31.353221+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[17385]: audit 2024-02-10T08:24:31.434704+0000 mon.a (mon.0) 675 : audit [DBG] from='client.? 172.21.15.67:0/3361541064' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[17385]: audit 2024-02-10T08:24:31.627658+0000 mgr.y (mgr.14547) 82 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[24488]: audit 2024-02-10T08:24:31.342378+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[24488]: audit 2024-02-10T08:24:31.344459+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[24488]: audit 2024-02-10T08:24:31.345742+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[24488]: audit 2024-02-10T08:24:31.353221+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[24488]: audit 2024-02-10T08:24:31.434704+0000 mon.a (mon.0) 675 : audit [DBG] from='client.? 172.21.15.67:0/3361541064' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T08:24:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:32 smithi067 bash[24488]: audit 2024-02-10T08:24:31.627658+0000 mgr.y (mgr.14547) 82 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:32 smithi134 bash[18722]: audit 2024-02-10T08:24:31.342378+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:32 smithi134 bash[18722]: audit 2024-02-10T08:24:31.344459+0000 mon.c (mon.2) 70 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:24:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:32 smithi134 bash[18722]: audit 2024-02-10T08:24:31.345742+0000 mon.c (mon.2) 71 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:24:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:32 smithi134 bash[18722]: audit 2024-02-10T08:24:31.353221+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:24:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:32 smithi134 bash[18722]: audit 2024-02-10T08:24:31.434704+0000 mon.a (mon.0) 675 : audit [DBG] from='client.? 172.21.15.67:0/3361541064' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T08:24:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:32 smithi134 bash[18722]: audit 2024-02-10T08:24:31.627658+0000 mgr.y (mgr.14547) 82 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:33 smithi134 bash[18722]: cluster 2024-02-10T08:24:32.570735+0000 mgr.y (mgr.14547) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:24:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:24:33.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:33 smithi067 bash[24488]: cluster 2024-02-10T08:24:32.570735+0000 mgr.y (mgr.14547) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:33 smithi067 bash[17385]: cluster 2024-02-10T08:24:32.570735+0000 mgr.y (mgr.14547) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:35 smithi134 bash[18722]: cluster 2024-02-10T08:24:34.571432+0000 mgr.y (mgr.14547) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:35 smithi067 bash[17385]: cluster 2024-02-10T08:24:34.571432+0000 mgr.y (mgr.14547) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:36.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:35 smithi067 bash[24488]: cluster 2024-02-10T08:24:34.571432+0000 mgr.y (mgr.14547) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:36.984 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:37.953 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:37] "GET /metrics HTTP/1.1" 200 172151 "" "Prometheus/2.33.4" 2024-02-10T08:24:37.954 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:37 smithi067 bash[17385]: cluster 2024-02-10T08:24:36.572825+0000 mgr.y (mgr.14547) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:37.954 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:37 smithi067 bash[24488]: cluster 2024-02-10T08:24:36.572825+0000 mgr.y (mgr.14547) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:37 smithi134 bash[18722]: cluster 2024-02-10T08:24:36.572825+0000 mgr.y (mgr.14547) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 -- 172.21.15.67:0/1742152003 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9400f95d0 msgr2=0x7fa940101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 --2- 172.21.15.67:0/1742152003 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9400f95d0 0x7fa940101ce0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa928004870 tx=0x7fa928009cb0).stop 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 -- 172.21.15.67:0/1742152003 shutdown_connections 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 --2- 172.21.15.67:0/1742152003 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9401021d0 0x7fa9401045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 --2- 172.21.15.67:0/1742152003 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9400f8c70 0x7fa9400f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 --2- 172.21.15.67:0/1742152003 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9400f95d0 0x7fa940101ce0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 -- 172.21.15.67:0/1742152003 >> 172.21.15.67:0/1742152003 conn(0x7fa9400f27a0 msgr2=0x7fa9400f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 -- 172.21.15.67:0/1742152003 shutdown_connections 2024-02-10T08:24:39.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.873+0000 7fa945aaf700 1 -- 172.21.15.67:0/1742152003 wait complete. 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 Processor -- start 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 -- start start 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9400f8c70 0x7fa94010e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9400f95d0 0x7fa94010e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 0x7fa9401082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa940108790 con 0x7fa9400f95d0 2024-02-10T08:24:39.880 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa940108900 con 0x7fa9400f8c70 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa940108ac0 con 0x7fa9401021d0 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 0x7fa9401082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa936ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9400f95d0 0x7fa94010e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 0x7fa9401082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:43640/0 (socket says 172.21.15.67:43640) 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 -- 172.21.15.67:0/2588700620 learned_addr learned my addr 172.21.15.67:0/2588700620 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93f7fe700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9400f8c70 0x7fa94010e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 -- 172.21.15.67:0/2588700620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9400f8c70 msgr2=0x7fa94010e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9400f8c70 0x7fa94010e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 -- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9400f95d0 msgr2=0x7fa94010e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9400f95d0 0x7fa94010e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:39.881 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 -- 172.21.15.67:0/2588700620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380097c0 con 0x7fa9401021d0 2024-02-10T08:24:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93ffff700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 0x7fa9401082a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa938001df0 tx=0x7fa938004c50).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9380099f0 con 0x7fa9401021d0 2024-02-10T08:24:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa9380090d0 con 0x7fa9401021d0 2024-02-10T08:24:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa938007b40 con 0x7fa9401021d0 2024-02-10T08:24:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa928005040 con 0x7fa9401021d0 2024-02-10T08:24:39.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.877+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9401090a0 con 0x7fa9401021d0 2024-02-10T08:24:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.881+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa940062100 con 0x7fa9401021d0 2024-02-10T08:24:39.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.881+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa93800fb30 con 0x7fa9401021d0 2024-02-10T08:24:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.881+0000 7fa93d775700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa92c061340 0x7fa92c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.881+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7fa93807d710 con 0x7fa9401021d0 2024-02-10T08:24:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.881+0000 7fa93f7fe700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa92c061340 0x7fa92c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.881+0000 7fa93f7fe700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa92c061340 0x7fa92c063800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa9300048b0 tx=0x7fa93000b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:39.890 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:39.889+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa93804d550 con 0x7fa9401021d0 2024-02-10T08:24:40.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:39 smithi134 bash[18722]: cluster 2024-02-10T08:24:38.573537+0000 mgr.y (mgr.14547) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:40.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.073+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa940106fc0 con 0x7fa9401021d0 2024-02-10T08:24:40.080 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.077+0000 7fa93d775700 1 -- 172.21.15.67:0/2588700620 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+21296 (secure 0 0 0) 0x7fa938050b70 con 0x7fa9401021d0 2024-02-10T08:24:40.082 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:24:40.082 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"fsid":"2dd26d0c-c7ec-11ee-95b8-87774f69a715","created":"2024-02-10T08:13:45.488251+0000","modified":"2024-02-10T08:24:01.783748+0000","last_up_change":"2024-02-10T08:22:20.247344+0000","last_in_change":"2024-02-10T08:21:54.660923+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-10T08:18:49.528063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-10T08:22:33.283700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-10T08:22:34.781350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T08:22:36.337577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T08:22:38.307932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T08:22:40.423655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"437dcceb-1970-491f-b22e-be8d9b063290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6803","nonce":2848595980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6805","nonce":2848595980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6809","nonce":2848595980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6807","nonce":2848595980}]},"public_addr":"172.21.15.67:6803/2848595980","cluster_addr":"172.21.15.67:6805/2848595980","heartbeat_back_addr":"172.21.15.67:6809/2848595980","heartbeat_front_addr":"172.21.15.67:6807/2848595980","state":["exists","up"]},{"osd":1,"uuid":"5233196f-2e05-44ce-a209-d6266a187402","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6811","nonce":1520735968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6813","nonce":1520735968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6817","nonce":1520735968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6815","nonce":1520735968}]},"public_addr":"172.21.15.67:6811/1520735968","cluster_addr":"172.21.15.67:6813/1520735968","heartbeat_back_addr":"172.21.15.67:6817/1520735968","heartbeat_front_addr":"172.21.15.67:6815/1520735968","state":["exists","up"]},{"osd":2,"uuid":"4b812fb6-4df3-47ad-a32d-5d84545750b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6819","nonce":1580907765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6821","nonce":1580907765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6825","nonce":1580907765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6823","nonce":1580907765}]},"public_addr":"172.21.15.67:6819/1580907765","cluster_addr":"172.21.15.67:6821/1580907765","heartbeat_back_addr":"172.21.15.67:6825/1580907765","heartbeat_front_addr":"172.21.15.67:6823/1580907765","state":["exists","up"]},{"osd":3,"uuid":"13ddc51e-07b3-4e88-a568-eb7ec0dedbad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6827","nonce":2711595830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6829","nonce":2711595830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6833","nonce":2711595830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6831","nonce":2711595830}]},"public_addr":"172.21.15.67:6827/2711595830","cluster_addr":"172.21.15.67:6829/2711595830","heartbeat_back_addr":"172.21.15.67:6833/2711595830","heartbeat_front_addr":"172.21.15.67:6831/2711595830","state":["exists","up"]},{"osd":4,"uuid":"c5a34154-4009-4d77-a6f1-7b1c6a7cdd12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6801","nonce":636669687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6803","nonce":636669687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6807","nonce":636669687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6805","nonce":636669687}]},"public_addr":"172.21.15.134:6801/636669687","cluster_addr":"172.21.15.134:6803/636669687","heartbeat_back_addr":"172.21.15.134:6807/636669687","heartbeat_front_addr":"172.21.15.134:6805/636669687","state":["exists","up"]},{"osd":5,"uuid":"5f79bc97-f76e-45d8-95e9-06654d75441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6809","nonce":2103042762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6811","nonce":2103042762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6815","nonce":2103042762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6813","nonce":2103042762}]},"public_addr":"172.21.15.134:6809/2103042762","cluster_addr":"172.21.15.134:6811/2103042762","heartbeat_back_addr":"172.21.15.134:6815/2103042762","heartbeat_front_addr":"172.21.15.134:6813/2103042762","state":["exists","up"]},{"osd":6,"uuid":"92ea5976-343a-4e4c-8b06-79ee579d4a84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6817","nonce":3982264053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6819","nonce":3982264053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6823","nonce":3982264053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6821","nonce":3982264053}]},"public_addr":"172.21.15.134:6817/3982264053","cluster_addr":"172.21.15.134:6819/3982264053","heartbeat_back_addr":"172.21.15.134:6823/3982264053","heartbeat_front_addr":"172.21.15.134:6821/3982264053","state":["exists","up"]},{"osd":7,"uuid":"4ebe0339-e352-47e1-adba-42e1cd076b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6825","nonce":1760805884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6827","nonce":1760805884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6831","nonce":1760805884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6829","nonce":1760805884}]},"public_addr":"172.21.15.134:6825/1760805884","cluster_addr":"172.21.15.134:6827/1760805884","heartbeat_back_addr":"172.21.15.134:6831/1760805884","heartbeat_front_addr":"172.21.15.134:6829/1760805884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:17:19.929877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:18:03.675497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:18:47.802905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:19:32.110706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:20:12.986423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:20:54.618613+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:21:36.412286+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:22:18.451049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/4022632212":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/502171530":"2024-02-11T08:23:00.509127+0000","172.21.15.67:6801/4022632212":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/430077928":"2024-02-11T08:14:44.793111+0000","172.21.15.67:6801/2933691832":"2024-02-11T08:14:44.793111+0000","172.21.15.67:6800/2933691832":"2024-02-11T08:14:44.793111+0000","172.21.15.67:0/3228839582":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/2063401242":"2024-02-11T08:14:14.841252+0000","172.21.15.67:0/1185089911":"2024-02-11T08:14:44.793111+0000","172.21.15.67:0/1039332145":"2024-02-11T08:14:14.841252+0000","172.21.15.67:6800/2812373217":"2024-02-11T08:14:14.841252+0000","172.21.15.67:0/2508290626":"2024-02-11T08:23:00.509127+0000","172.21.15.67:6801/2812373217":"2024-02-11T08:14:14.841252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa92c061340 msgr2=0x7fa92c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa92c061340 0x7fa92c063800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fa9300048b0 tx=0x7fa93000b040).stop 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 msgr2=0x7fa9401082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 0x7fa9401082a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa938001df0 tx=0x7fa938004c50).stop 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 shutdown_connections 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa92c061340 0x7fa92c063800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa9401021d0 0x7fa9401082a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9400f8c70 0x7fa94010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 --2- 172.21.15.67:0/2588700620 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa9400f95d0 0x7fa94010e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 >> 172.21.15.67:0/2588700620 conn(0x7fa9400f27a0 msgr2=0x7fa9400f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:40.085 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 shutdown_connections 2024-02-10T08:24:40.086 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:40.081+0000 7fa945aaf700 1 -- 172.21.15.67:0/2588700620 wait complete. 2024-02-10T08:24:40.115 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:39 smithi067 bash[17385]: cluster 2024-02-10T08:24:38.573537+0000 mgr.y (mgr.14547) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:40.115 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:39 smithi067 bash[24488]: cluster 2024-02-10T08:24:38.573537+0000 mgr.y (mgr.14547) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:40.769 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-10T08:24:40.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T08:24:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:40 smithi134 bash[18722]: audit 2024-02-10T08:24:40.078740+0000 mon.c (mon.2) 72 : audit [DBG] from='client.? 172.21.15.67:0/2588700620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:24:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:40 smithi067 bash[17385]: audit 2024-02-10T08:24:40.078740+0000 mon.c (mon.2) 72 : audit [DBG] from='client.? 172.21.15.67:0/2588700620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:24:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:40 smithi067 bash[24488]: audit 2024-02-10T08:24:40.078740+0000 mon.c (mon.2) 72 : audit [DBG] from='client.? 172.21.15.67:0/2588700620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:24:42.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:41 smithi134 bash[18722]: cluster 2024-02-10T08:24:40.574226+0000 mgr.y (mgr.14547) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:42.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:24:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:24:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:41 smithi067 bash[17385]: cluster 2024-02-10T08:24:40.574226+0000 mgr.y (mgr.14547) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:42.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:41 smithi067 bash[24488]: cluster 2024-02-10T08:24:40.574226+0000 mgr.y (mgr.14547) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:43.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:42 smithi134 bash[18722]: audit 2024-02-10T08:24:41.639098+0000 mgr.y (mgr.14547) 88 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:42 smithi067 bash[17385]: audit 2024-02-10T08:24:41.639098+0000 mgr.y (mgr.14547) 88 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:43.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:42 smithi067 bash[24488]: audit 2024-02-10T08:24:41.639098+0000 mgr.y (mgr.14547) 88 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:43.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:24:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:24:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:43 smithi134 bash[18722]: cluster 2024-02-10T08:24:42.575683+0000 mgr.y (mgr.14547) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:43 smithi067 bash[17385]: cluster 2024-02-10T08:24:42.575683+0000 mgr.y (mgr.14547) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:44.134 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:43 smithi067 bash[24488]: cluster 2024-02-10T08:24:42.575683+0000 mgr.y (mgr.14547) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:45.560 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:45 smithi067 bash[17385]: cluster 2024-02-10T08:24:44.576414+0000 mgr.y (mgr.14547) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:45.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:45 smithi067 bash[24488]: cluster 2024-02-10T08:24:44.576414+0000 mgr.y (mgr.14547) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:46.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:45 smithi134 bash[18722]: cluster 2024-02-10T08:24:44.576414+0000 mgr.y (mgr.14547) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:46.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:46 smithi067 bash[17385]: cluster 2024-02-10T08:24:46.577190+0000 mgr.y (mgr.14547) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:46.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:46 smithi067 bash[24488]: cluster 2024-02-10T08:24:46.577190+0000 mgr.y (mgr.14547) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:46 smithi134 bash[18722]: cluster 2024-02-10T08:24:46.577190+0000 mgr.y (mgr.14547) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:47] "GET /metrics HTTP/1.1" 200 172148 "" "Prometheus/2.33.4" 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 -- 172.21.15.67:0/926141101 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480fd4f0 msgr2=0x7f9b480f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 --2- 172.21.15.67:0/926141101 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480fd4f0 0x7f9b480f6d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f9b300047b0 tx=0x7f9b30009cb0).stop 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 -- 172.21.15.67:0/926141101 shutdown_connections 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 --2- 172.21.15.67:0/926141101 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480f7400 0x7f9b48103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 --2- 172.21.15.67:0/926141101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480fcb90 0x7f9b480fcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 --2- 172.21.15.67:0/926141101 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480fd4f0 0x7f9b480f6d20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 -- 172.21.15.67:0/926141101 >> 172.21.15.67:0/926141101 conn(0x7f9b480f27a0 msgr2=0x7f9b480f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 -- 172.21.15.67:0/926141101 shutdown_connections 2024-02-10T08:24:48.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.481+0000 7f9b4d753700 1 -- 172.21.15.67:0/926141101 wait complete. 2024-02-10T08:24:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 Processor -- start 2024-02-10T08:24:48.487 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 -- start start 2024-02-10T08:24:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480f7400 0x7f9b48109030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 0x7f9b48109520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480fd4f0 0x7f9b48109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48114220 con 0x7f9b480fcb90 2024-02-10T08:24:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48114390 con 0x7f9b480fd4f0 2024-02-10T08:24:48.488 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48114550 con 0x7f9b480f7400 2024-02-10T08:24:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 0x7f9b48109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b46ffd700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480f7400 0x7f9b48109030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 0x7f9b48109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:52122/0 (socket says 172.21.15.67:52122) 2024-02-10T08:24:48.489 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 -- 172.21.15.67:0/1615663286 learned_addr learned my addr 172.21.15.67:0/1615663286 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b477fe700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480fd4f0 0x7f9b48109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 -- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480f7400 msgr2=0x7f9b48109030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480f7400 0x7f9b48109030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 -- 172.21.15.67:0/1615663286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480fd4f0 msgr2=0x7f9b48109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480fd4f0 0x7f9b48109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 -- 172.21.15.67:0/1615663286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400097c0 con 0x7f9b480fcb90 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b46ffd700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480f7400 0x7f9b48109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:24:48.490 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b3e7fc700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 0x7f9b48109520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f9b30000e50 tx=0x7f9b30009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30018070 con 0x7f9b480fcb90 2024-02-10T08:24:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f9b30013e70 con 0x7f9b480fcb90 2024-02-10T08:24:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b30005040 con 0x7f9b480fcb90 2024-02-10T08:24:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3001db30 con 0x7f9b480fcb90 2024-02-10T08:24:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.485+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b48114a70 con 0x7f9b480fcb90 2024-02-10T08:24:48.491 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.489+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48060740 con 0x7f9b480fcb90 2024-02-10T08:24:48.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.489+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f9b30009240 con 0x7f9b480fcb90 2024-02-10T08:24:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.489+0000 7f9b44ff9700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f9b34061340 0x7f9b34063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:48.493 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.489+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f9b3007e2c0 con 0x7f9b480fcb90 2024-02-10T08:24:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.489+0000 7f9b46ffd700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f9b34061340 0x7f9b34063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:48.494 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.489+0000 7f9b46ffd700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f9b34061340 0x7f9b34063800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9b380103c0 tx=0x7f9b3800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:48.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.497+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f9b3004e040 con 0x7f9b480fcb90 2024-02-10T08:24:48.688 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.685+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b480475e0 con 0x7f9b480fcb90 2024-02-10T08:24:48.690 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b44ff9700 1 -- 172.21.15.67:0/1615663286 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+21296 (secure 0 0 0) 0x7f9b30051660 con 0x7f9b480fcb90 2024-02-10T08:24:48.691 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:24:48.691 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":63,"fsid":"2dd26d0c-c7ec-11ee-95b8-87774f69a715","created":"2024-02-10T08:13:45.488251+0000","modified":"2024-02-10T08:24:01.783748+0000","last_up_change":"2024-02-10T08:22:20.247344+0000","last_in_change":"2024-02-10T08:21:54.660923+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-10T08:18:49.528063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-10T08:22:33.283700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-10T08:22:34.781350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-10T08:22:36.337577+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-10T08:22:38.307932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-10T08:22:40.423655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"437dcceb-1970-491f-b22e-be8d9b063290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6803","nonce":2848595980}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6805","nonce":2848595980}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6809","nonce":2848595980}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":2848595980},{"type":"v1","addr":"172.21.15.67:6807","nonce":2848595980}]},"public_addr":"172.21.15.67:6803/2848595980","cluster_addr":"172.21.15.67:6805/2848595980","heartbeat_back_addr":"172.21.15.67:6809/2848595980","heartbeat_front_addr":"172.21.15.67:6807/2848595980","state":["exists","up"]},{"osd":1,"uuid":"5233196f-2e05-44ce-a209-d6266a187402","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6811","nonce":1520735968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6813","nonce":1520735968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6817","nonce":1520735968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1520735968},{"type":"v1","addr":"172.21.15.67:6815","nonce":1520735968}]},"public_addr":"172.21.15.67:6811/1520735968","cluster_addr":"172.21.15.67:6813/1520735968","heartbeat_back_addr":"172.21.15.67:6817/1520735968","heartbeat_front_addr":"172.21.15.67:6815/1520735968","state":["exists","up"]},{"osd":2,"uuid":"4b812fb6-4df3-47ad-a32d-5d84545750b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6819","nonce":1580907765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6821","nonce":1580907765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6825","nonce":1580907765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1580907765},{"type":"v1","addr":"172.21.15.67:6823","nonce":1580907765}]},"public_addr":"172.21.15.67:6819/1580907765","cluster_addr":"172.21.15.67:6821/1580907765","heartbeat_back_addr":"172.21.15.67:6825/1580907765","heartbeat_front_addr":"172.21.15.67:6823/1580907765","state":["exists","up"]},{"osd":3,"uuid":"13ddc51e-07b3-4e88-a568-eb7ec0dedbad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6827","nonce":2711595830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6829","nonce":2711595830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6832","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6833","nonce":2711595830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":2711595830},{"type":"v1","addr":"172.21.15.67:6831","nonce":2711595830}]},"public_addr":"172.21.15.67:6827/2711595830","cluster_addr":"172.21.15.67:6829/2711595830","heartbeat_back_addr":"172.21.15.67:6833/2711595830","heartbeat_front_addr":"172.21.15.67:6831/2711595830","state":["exists","up"]},{"osd":4,"uuid":"c5a34154-4009-4d77-a6f1-7b1c6a7cdd12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6801","nonce":636669687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6803","nonce":636669687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6807","nonce":636669687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":636669687},{"type":"v1","addr":"172.21.15.134:6805","nonce":636669687}]},"public_addr":"172.21.15.134:6801/636669687","cluster_addr":"172.21.15.134:6803/636669687","heartbeat_back_addr":"172.21.15.134:6807/636669687","heartbeat_front_addr":"172.21.15.134:6805/636669687","state":["exists","up"]},{"osd":5,"uuid":"5f79bc97-f76e-45d8-95e9-06654d75441a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6809","nonce":2103042762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6811","nonce":2103042762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6815","nonce":2103042762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":2103042762},{"type":"v1","addr":"172.21.15.134:6813","nonce":2103042762}]},"public_addr":"172.21.15.134:6809/2103042762","cluster_addr":"172.21.15.134:6811/2103042762","heartbeat_back_addr":"172.21.15.134:6815/2103042762","heartbeat_front_addr":"172.21.15.134:6813/2103042762","state":["exists","up"]},{"osd":6,"uuid":"92ea5976-343a-4e4c-8b06-79ee579d4a84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6817","nonce":3982264053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6819","nonce":3982264053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6823","nonce":3982264053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":3982264053},{"type":"v1","addr":"172.21.15.134:6821","nonce":3982264053}]},"public_addr":"172.21.15.134:6817/3982264053","cluster_addr":"172.21.15.134:6819/3982264053","heartbeat_back_addr":"172.21.15.134:6823/3982264053","heartbeat_front_addr":"172.21.15.134:6821/3982264053","state":["exists","up"]},{"osd":7,"uuid":"4ebe0339-e352-47e1-adba-42e1cd076b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6825","nonce":1760805884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6827","nonce":1760805884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6831","nonce":1760805884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":1760805884},{"type":"v1","addr":"172.21.15.134:6829","nonce":1760805884}]},"public_addr":"172.21.15.134:6825/1760805884","cluster_addr":"172.21.15.134:6827/1760805884","heartbeat_back_addr":"172.21.15.134:6831/1760805884","heartbeat_front_addr":"172.21.15.134:6829/1760805884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:17:19.929877+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:18:03.675497+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:18:47.802905+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:19:32.110706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:20:12.986423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:20:54.618613+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:21:36.412286+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T08:22:18.451049+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:6800/4022632212":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/502171530":"2024-02-11T08:23:00.509127+0000","172.21.15.67:6801/4022632212":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/430077928":"2024-02-11T08:14:44.793111+0000","172.21.15.67:6801/2933691832":"2024-02-11T08:14:44.793111+0000","172.21.15.67:6800/2933691832":"2024-02-11T08:14:44.793111+0000","172.21.15.67:0/3228839582":"2024-02-11T08:23:00.509127+0000","172.21.15.67:0/2063401242":"2024-02-11T08:14:14.841252+0000","172.21.15.67:0/1185089911":"2024-02-11T08:14:44.793111+0000","172.21.15.67:0/1039332145":"2024-02-11T08:14:14.841252+0000","172.21.15.67:6800/2812373217":"2024-02-11T08:14:14.841252+0000","172.21.15.67:0/2508290626":"2024-02-11T08:23:00.509127+0000","172.21.15.67:6801/2812373217":"2024-02-11T08:14:14.841252+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T08:24:48.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f9b34061340 msgr2=0x7f9b34063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:48.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f9b34061340 0x7f9b34063800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9b380103c0 tx=0x7f9b3800b040).stop 2024-02-10T08:24:48.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 msgr2=0x7f9b48109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:48.693 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 0x7f9b48109520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f9b30000e50 tx=0x7f9b30009cb0).stop 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 shutdown_connections 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f9b34061340 0x7f9b34063800 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b480fd4f0 0x7f9b48109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f9b480f7400 0x7f9b48109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 --2- 172.21.15.67:0/1615663286 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f9b480fcb90 0x7f9b48109520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 >> 172.21.15.67:0/1615663286 conn(0x7f9b480f27a0 msgr2=0x7f9b480fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 shutdown_connections 2024-02-10T08:24:48.694 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:48.689+0000 7f9b4d753700 1 -- 172.21.15.67:0/1615663286 wait complete. 2024-02-10T08:24:49.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-10T08:24:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:49 smithi067 bash[17385]: cluster 2024-02-10T08:24:48.577799+0000 mgr.y (mgr.14547) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:49 smithi067 bash[17385]: audit 2024-02-10T08:24:48.688673+0000 mon.a (mon.0) 676 : audit [DBG] from='client.? 172.21.15.67:0/1615663286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:24:49.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:49 smithi067 bash[24488]: cluster 2024-02-10T08:24:48.577799+0000 mgr.y (mgr.14547) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:49.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:49 smithi067 bash[24488]: audit 2024-02-10T08:24:48.688673+0000 mon.a (mon.0) 676 : audit [DBG] from='client.? 172.21.15.67:0/1615663286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:24:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:49 smithi134 bash[18722]: cluster 2024-02-10T08:24:48.577799+0000 mgr.y (mgr.14547) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:49 smithi134 bash[18722]: audit 2024-02-10T08:24:48.688673+0000 mon.a (mon.0) 676 : audit [DBG] from='client.? 172.21.15.67:0/1615663286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T08:24:52.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:51 smithi134 bash[18722]: cluster 2024-02-10T08:24:50.578495+0000 mgr.y (mgr.14547) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:52.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:24:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:24:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:51 smithi067 bash[17385]: cluster 2024-02-10T08:24:50.578495+0000 mgr.y (mgr.14547) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:51 smithi067 bash[24488]: cluster 2024-02-10T08:24:50.578495+0000 mgr.y (mgr.14547) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:52 smithi134 bash[18722]: audit 2024-02-10T08:24:51.647900+0000 mgr.y (mgr.14547) 94 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:53.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:52 smithi067 bash[17385]: audit 2024-02-10T08:24:51.647900+0000 mgr.y (mgr.14547) 94 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:52 smithi067 bash[24488]: audit 2024-02-10T08:24:51.647900+0000 mgr.y (mgr.14547) 94 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:24:53.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:24:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:24:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:53 smithi134 bash[18722]: cluster 2024-02-10T08:24:52.579979+0000 mgr.y (mgr.14547) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:53 smithi067 bash[17385]: cluster 2024-02-10T08:24:52.579979+0000 mgr.y (mgr.14547) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:54.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:53 smithi067 bash[24488]: cluster 2024-02-10T08:24:52.579979+0000 mgr.y (mgr.14547) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:54.207 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:24:55.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:55 smithi067 bash[24488]: cluster 2024-02-10T08:24:54.580432+0000 mgr.y (mgr.14547) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:55 smithi067 bash[17385]: cluster 2024-02-10T08:24:54.580432+0000 mgr.y (mgr.14547) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:56.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:55 smithi134 bash[18722]: cluster 2024-02-10T08:24:54.580432+0000 mgr.y (mgr.14547) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:24:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 -- 172.21.15.67:0/1003504084 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 msgr2=0x7f8040101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 --2- 172.21.15.67:0/1003504084 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 0x7f8040101d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8030004870 tx=0x7f8030009cb0).stop 2024-02-10T08:24:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 -- 172.21.15.67:0/1003504084 shutdown_connections 2024-02-10T08:24:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 --2- 172.21.15.67:0/1003504084 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8040102340 0x7f8040104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 --2- 172.21.15.67:0/1003504084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80400f8e70 0x7f80400f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:57.070 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 --2- 172.21.15.67:0/1003504084 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 0x7f8040101d20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 -- 172.21.15.67:0/1003504084 >> 172.21.15.67:0/1003504084 conn(0x7f80400f27e0 msgr2=0x7f80400f4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 -- 172.21.15.67:0/1003504084 shutdown_connections 2024-02-10T08:24:57.071 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.065+0000 7f8047c41700 1 -- 172.21.15.67:0/1003504084 wait complete. 2024-02-10T08:24:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 Processor -- start 2024-02-10T08:24:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 -- start start 2024-02-10T08:24:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80400f8e70 0x7f80400f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 0x7f80400f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8040102340 0x7f80400f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401165d0 con 0x7f80400f8e70 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040116740 con 0x7f8040102340 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8040116900 con 0x7f80400f97d0 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80400f8e70 0x7f80400f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80400f8e70 0x7f80400f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:50630/0 (socket says 172.21.15.67:50630) 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 -- 172.21.15.67:0/2059965899 learned_addr learned my addr 172.21.15.67:0/2059965899 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80451dc700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 0x7f80400f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80461de700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8040102340 0x7f80400f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 -- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 msgr2=0x7f80400f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f80400f97d0 0x7f80400f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 -- 172.21.15.67:0/2059965899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8040102340 msgr2=0x7f80400f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8040102340 0x7f80400f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 -- 172.21.15.67:0/2059965899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030005040 con 0x7f80400f8e70 2024-02-10T08:24:57.073 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80461de700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8040102340 0x7f80400f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:24:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f80459dd700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80400f8e70 0x7f80400f6c60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f803c00db50 tx=0x7f803c004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0077a0 con 0x7f80400f8e70 2024-02-10T08:24:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f803c00f040 con 0x7f80400f8e70 2024-02-10T08:24:57.074 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8040116be0 con 0x7f80400f8e70 2024-02-10T08:24:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0139a0 con 0x7f80400f8e70 2024-02-10T08:24:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.069+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8040117120 con 0x7f80400f8e70 2024-02-10T08:24:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8040059400 con 0x7f80400f8e70 2024-02-10T08:24:57.075 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f803c00b750 con 0x7f80400f8e70 2024-02-10T08:24:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f802c061340 0x7f802c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:57.076 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f803c07cb30 con 0x7f80400f8e70 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f80451dc700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f802c061340 0x7f802c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] conn(0x7f802c06b1d0 0x7f802c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 --> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f802c06dcf0 con 0x7f802c06b1d0 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f803c081220 con 0x7f80400f8e70 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f80461de700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] conn(0x7f802c06b1d0 0x7f802c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f80451dc700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f802c061340 0x7f802c063800 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f80400f8400 tx=0x7f803000b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:57.077 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f80461de700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] conn(0x7f802c06b1d0 0x7f802c06d670 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:24:57.078 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.073+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== osd.0 v2:172.21.15.67:6802/2848595980 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f802c06dcf0 con 0x7f802c06b1d0 2024-02-10T08:24:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 --> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80400475e0 con 0x7f802c06b1d0 2024-02-10T08:24:57.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8036ffd700 1 -- 172.21.15.67:0/2059965899 <== osd.0 v2:172.21.15.67:6802/2848595980 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f80400475e0 con 0x7f802c06b1d0 2024-02-10T08:24:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] conn(0x7f802c06b1d0 msgr2=0x7f802c06d670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6802/2848595980,v1:172.21.15.67:6803/2848595980] conn(0x7f802c06b1d0 0x7f802c06d670 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:24:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f802c061340 msgr2=0x7f802c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f802c061340 0x7f802c063800 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f80400f8400 tx=0x7f803000b040).stop 2024-02-10T08:24:57.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80400f8e70 msgr2=0x7f80400f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:24:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 --2- 172.21.15.67:0/2059965899 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f80400f8e70 0x7f80400f6c60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f803c00db50 tx=0x7f803c004ee0).stop 2024-02-10T08:24:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f80461de700 1 -- 172.21.15.67:0/2059965899 reap_dead start 2024-02-10T08:24:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 shutdown_connections 2024-02-10T08:24:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 >> 172.21.15.67:0/2059965899 conn(0x7f80400f27e0 msgr2=0x7f80400f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:24:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 shutdown_connections 2024-02-10T08:24:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:24:57.093+0000 7f8047c41700 1 -- 172.21.15.67:0/2059965899 wait complete. 2024-02-10T08:24:57.878 INFO:teuthology.orchestra.run.smithi067.stdout:34359738462 2024-02-10T08:24:57.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-10T08:24:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:57 smithi134 bash[18722]: cluster 2024-02-10T08:24:56.581801+0000 mgr.y (mgr.14547) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:57 smithi067 bash[17385]: cluster 2024-02-10T08:24:56.581801+0000 mgr.y (mgr.14547) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:57 smithi067 bash[24488]: cluster 2024-02-10T08:24:56.581801+0000 mgr.y (mgr.14547) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:24:58.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:24:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:24:57] "GET /metrics HTTP/1.1" 200 172148 "" "Prometheus/2.33.4" 2024-02-10T08:25:00.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:24:59 smithi134 bash[18722]: cluster 2024-02-10T08:24:58.582627+0000 mgr.y (mgr.14547) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:00.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:24:59 smithi067 bash[24488]: cluster 2024-02-10T08:24:58.582627+0000 mgr.y (mgr.14547) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:00.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:24:59 smithi067 bash[17385]: cluster 2024-02-10T08:24:58.582627+0000 mgr.y (mgr.14547) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:01 smithi067 bash[17385]: cluster 2024-02-10T08:25:00.583315+0000 mgr.y (mgr.14547) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:01.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:01 smithi067 bash[24488]: cluster 2024-02-10T08:25:00.583315+0000 mgr.y (mgr.14547) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:01.984 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:01 smithi134 bash[18722]: cluster 2024-02-10T08:25:00.583315+0000 mgr.y (mgr.14547) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:01.985 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:25:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:25:02.667 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:03.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:02 smithi134 bash[18722]: audit 2024-02-10T08:25:01.658879+0000 mgr.y (mgr.14547) 100 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:02 smithi067 bash[17385]: audit 2024-02-10T08:25:01.658879+0000 mgr.y (mgr.14547) 100 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:03.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:02 smithi067 bash[24488]: audit 2024-02-10T08:25:01.658879+0000 mgr.y (mgr.14547) 100 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:03.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:25:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:25:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:03 smithi067 bash[17385]: cluster 2024-02-10T08:25:02.584778+0000 mgr.y (mgr.14547) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:03.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:03 smithi067 bash[24488]: cluster 2024-02-10T08:25:02.584778+0000 mgr.y (mgr.14547) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:04.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:03 smithi134 bash[18722]: cluster 2024-02-10T08:25:02.584778+0000 mgr.y (mgr.14547) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:05.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:04 smithi134 bash[18722]: cluster 2024-02-10T08:25:04.585227+0000 mgr.y (mgr.14547) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:04 smithi067 bash[17385]: cluster 2024-02-10T08:25:04.585227+0000 mgr.y (mgr.14547) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:05.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:04 smithi067 bash[24488]: cluster 2024-02-10T08:25:04.585227+0000 mgr.y (mgr.14547) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:05.376 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- 172.21.15.67:0/390544962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 msgr2=0x7ffb6c101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- 172.21.15.67:0/390544962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c101d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7ffb60004870 tx=0x7ffb60009cb0).stop 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- 172.21.15.67:0/390544962 shutdown_connections 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- 172.21.15.67:0/390544962 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb6c1021f0 0x7ffb6c1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- 172.21.15.67:0/390544962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c0f8c90 0x7ffb6c0f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- 172.21.15.67:0/390544962 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c101d00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- 172.21.15.67:0/390544962 >> 172.21.15.67:0/390544962 conn(0x7ffb6c0f27a0 msgr2=0x7ffb6c0f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- 172.21.15.67:0/390544962 shutdown_connections 2024-02-10T08:25:05.377 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- 172.21.15.67:0/390544962 wait complete. 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 Processor -- start 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- start start 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c0f8c90 0x7ffb6c110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb6c1021f0 0x7ffb6c10a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10a7d0 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10a940 con 0x7ffb6c0f8c90 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb72a49700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10ab00 con 0x7ffb6c1021f0 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:05.379 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c110630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57064/0 (socket says 172.21.15.67:57064) 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 -- 172.21.15.67:0/1238756714 learned_addr learned my addr 172.21.15.67:0/1238756714 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb70fe6700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb6c1021f0 0x7ffb6c10a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6bfff700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c0f8c90 0x7ffb6c110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 -- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb6c1021f0 msgr2=0x7ffb6c10a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7ffb6c1021f0 0x7ffb6c10a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 -- 172.21.15.67:0/1238756714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c0f8c90 msgr2=0x7ffb6c110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.373+0000 7ffb6b7fe700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6c0f8c90 0x7ffb6c110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb6b7fe700 1 -- 172.21.15.67:0/1238756714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60005040 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.380 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb6b7fe700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c110630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7ffb60005010 tx=0x7ffb60009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000e030 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7ffb600096d0 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.381 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c10ad80 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60018070 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c116f70 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb6c0f8510 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.382 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7ffb60009240 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb697fa700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7ffb54061340 0x7ffb54063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.377+0000 7ffb6bfff700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7ffb54061340 0x7ffb54063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7ffb6007e000 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb697fa700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] conn(0x7ffb5406b250 0x7ffb5406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 --> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb5406dd70 con 0x7ffb5406b250 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb60080650 con 0x7ffb6c0f95f0 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb6bfff700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7ffb54061340 0x7ffb54063800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ffb5c004dd0 tx=0x7ffb5c00b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:05.383 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb70fe6700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] conn(0x7ffb5406b250 0x7ffb5406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb70fe6700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] conn(0x7ffb5406b250 0x7ffb5406d6f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:05.384 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.381+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== osd.1 v2:172.21.15.67:6810/1520735968 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ffb5406dd70 con 0x7ffb5406b250 2024-02-10T08:25:05.402 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.397+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 --> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb6c060740 con 0x7ffb5406b250 2024-02-10T08:25:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb697fa700 1 -- 172.21.15.67:0/1238756714 <== osd.1 v2:172.21.15.67:6810/1520735968 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb6c060740 con 0x7ffb5406b250 2024-02-10T08:25:05.403 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] conn(0x7ffb5406b250 msgr2=0x7ffb5406d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6810/1520735968,v1:172.21.15.67:6811/1520735968] conn(0x7ffb5406b250 0x7ffb5406d6f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7ffb54061340 msgr2=0x7ffb54063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7ffb54061340 0x7ffb54063800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ffb5c004dd0 tx=0x7ffb5c00b040).stop 2024-02-10T08:25:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 msgr2=0x7ffb6c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 --2- 172.21.15.67:0/1238756714 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7ffb6c0f95f0 0x7ffb6c110630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7ffb60005010 tx=0x7ffb60009cb0).stop 2024-02-10T08:25:05.404 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb70fe6700 1 -- 172.21.15.67:0/1238756714 reap_dead start 2024-02-10T08:25:05.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 shutdown_connections 2024-02-10T08:25:05.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 >> 172.21.15.67:0/1238756714 conn(0x7ffb6c0f27a0 msgr2=0x7ffb6c0f4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:05.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 shutdown_connections 2024-02-10T08:25:05.405 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:05.401+0000 7ffb72a49700 1 -- 172.21.15.67:0/1238756714 wait complete. 2024-02-10T08:25:06.128 INFO:teuthology.orchestra.run.smithi067.stdout:55834574934 2024-02-10T08:25:06.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-10T08:25:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:07 smithi134 bash[18722]: cluster 2024-02-10T08:25:06.586562+0000 mgr.y (mgr.14547) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:07 smithi067 bash[17385]: cluster 2024-02-10T08:25:06.586562+0000 mgr.y (mgr.14547) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:25:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:07] "GET /metrics HTTP/1.1" 200 172151 "" "Prometheus/2.33.4" 2024-02-10T08:25:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:07 smithi067 bash[24488]: cluster 2024-02-10T08:25:06.586562+0000 mgr.y (mgr.14547) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:09 smithi134 bash[18722]: cluster 2024-02-10T08:25:08.587305+0000 mgr.y (mgr.14547) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:09 smithi067 bash[17385]: cluster 2024-02-10T08:25:08.587305+0000 mgr.y (mgr.14547) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:09 smithi067 bash[24488]: cluster 2024-02-10T08:25:08.587305+0000 mgr.y (mgr.14547) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:10.913 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:11.983 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:11 smithi067 bash[17385]: cluster 2024-02-10T08:25:10.588138+0000 mgr.y (mgr.14547) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:11.983 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:11 smithi067 bash[24488]: cluster 2024-02-10T08:25:10.588138+0000 mgr.y (mgr.14547) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:11 smithi134 bash[18722]: cluster 2024-02-10T08:25:10.588138+0000 mgr.y (mgr.14547) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:12.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:25:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:25:13.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:25:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:25:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:13 smithi134 bash[18722]: audit 2024-02-10T08:25:11.667524+0000 mgr.y (mgr.14547) 106 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:13 smithi134 bash[18722]: cluster 2024-02-10T08:25:12.588957+0000 mgr.y (mgr.14547) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:13.532 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:13 smithi067 bash[24488]: audit 2024-02-10T08:25:11.667524+0000 mgr.y (mgr.14547) 106 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:13.532 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:13 smithi067 bash[24488]: cluster 2024-02-10T08:25:12.588957+0000 mgr.y (mgr.14547) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:13.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:13 smithi067 bash[17385]: audit 2024-02-10T08:25:11.667524+0000 mgr.y (mgr.14547) 106 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:13.532 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:13 smithi067 bash[17385]: cluster 2024-02-10T08:25:12.588957+0000 mgr.y (mgr.14547) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 -- 172.21.15.67:0/3580995510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab8104490 msgr2=0x7f8ab8106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 --2- 172.21.15.67:0/3580995510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab8104490 0x7f8ab8106880 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8ab0004870 tx=0x7f8ab0009cb0).stop 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 -- 172.21.15.67:0/3580995510 shutdown_connections 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 --2- 172.21.15.67:0/3580995510 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab8104490 0x7f8ab8106880 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 --2- 172.21.15.67:0/3580995510 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8ab806b210 0x7f8ab806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 --2- 172.21.15.67:0/3580995510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab806bae0 0x7f8ab8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.729+0000 7f8abef80700 1 -- 172.21.15.67:0/3580995510 >> 172.21.15.67:0/3580995510 conn(0x7f8ab80f3810 msgr2=0x7f8ab80f5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- 172.21.15.67:0/3580995510 shutdown_connections 2024-02-10T08:25:13.735 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- 172.21.15.67:0/3580995510 wait complete. 2024-02-10T08:25:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 Processor -- start 2024-02-10T08:25:13.736 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- start start 2024-02-10T08:25:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab806b210 0x7f8ab8110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab806bae0 0x7f8ab8110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8ab8104490 0x7f8ab810a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab810a730 con 0x7f8ab806b210 2024-02-10T08:25:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab810a8a0 con 0x7f8ab806bae0 2024-02-10T08:25:13.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab810aa10 con 0x7f8ab8104490 2024-02-10T08:25:13.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abcd1c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab806b210 0x7f8ab8110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abcd1c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab806b210 0x7f8ab8110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34062/0 (socket says 172.21.15.67:34062) 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abcd1c700 1 -- 172.21.15.67:0/628384140 learned_addr learned my addr 172.21.15.67:0/628384140 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8ab8104490 0x7f8ab810a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8ab7fff700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab806bae0 0x7f8ab8110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 -- 172.21.15.67:0/628384140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab806bae0 msgr2=0x7f8ab8110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab806bae0 0x7f8ab8110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 -- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab806b210 msgr2=0x7f8ab8110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab806b210 0x7f8ab8110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:13.739 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 -- 172.21.15.67:0/628384140 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0005040 con 0x7f8ab8104490 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abcd1c700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8ab806b210 0x7f8ab8110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abd51d700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8ab8104490 0x7f8ab810a240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f8ab8061f80 tx=0x7f8ab0009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000e030 con 0x7f8ab8104490 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.733+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab810ac90 con 0x7f8ab8104490 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f8ab0013660 con 0x7f8ab8104490 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab8116ed0 con 0x7f8ab8104490 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0018070 con 0x7f8ab8104490 2024-02-10T08:25:13.740 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ab80f5d70 con 0x7f8ab8104490 2024-02-10T08:25:13.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8ab0009450 con 0x7f8ab8104490 2024-02-10T08:25:13.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8aa0061340 0x7f8aa0063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:13.741 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f8ab007e000 con 0x7f8ab8104490 2024-02-10T08:25:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8abcd1c700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8aa0061340 0x7f8aa0063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] conn(0x7f8aa006b250 0x7f8aa006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 --> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8aa006dd70 con 0x7f8aa006b250 2024-02-10T08:25:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ab0080650 con 0x7f8ab8104490 2024-02-10T08:25:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8ab7fff700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] conn(0x7f8aa006b250 0x7f8aa006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:13.742 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.737+0000 7f8abcd1c700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8aa0061340 0x7f8aa0063800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8aa8007d30 tx=0x7f8aa800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.741+0000 7f8ab7fff700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] conn(0x7f8aa006b250 0x7f8aa006d6f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:13.743 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.741+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== osd.2 v2:172.21.15.67:6818/1580907765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8aa006dd70 con 0x7f8aa006b250 2024-02-10T08:25:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.757+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 --> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ab805fa50 con 0x7f8aa006b250 2024-02-10T08:25:13.763 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8ab5ffb700 1 -- 172.21.15.67:0/628384140 <== osd.2 v2:172.21.15.67:6818/1580907765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ab805fa50 con 0x7f8aa006b250 2024-02-10T08:25:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] conn(0x7f8aa006b250 msgr2=0x7f8aa006d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6818/1580907765,v1:172.21.15.67:6819/1580907765] conn(0x7f8aa006b250 0x7f8aa006d6f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8aa0061340 msgr2=0x7f8aa0063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8aa0061340 0x7f8aa0063800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8aa8007d30 tx=0x7f8aa800b040).stop 2024-02-10T08:25:13.764 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8ab8104490 msgr2=0x7f8ab810a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 --2- 172.21.15.67:0/628384140 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8ab8104490 0x7f8ab810a240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f8ab8061f80 tx=0x7f8ab0009cb0).stop 2024-02-10T08:25:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abd51d700 1 -- 172.21.15.67:0/628384140 reap_dead start 2024-02-10T08:25:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 shutdown_connections 2024-02-10T08:25:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 >> 172.21.15.67:0/628384140 conn(0x7f8ab80f3810 msgr2=0x7f8ab80f4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 shutdown_connections 2024-02-10T08:25:13.765 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:13.761+0000 7f8abef80700 1 -- 172.21.15.67:0/628384140 wait complete. 2024-02-10T08:25:14.577 INFO:teuthology.orchestra.run.smithi067.stdout:77309411406 2024-02-10T08:25:14.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-10T08:25:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:15 smithi134 bash[18722]: cluster 2024-02-10T08:25:14.589825+0000 mgr.y (mgr.14547) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:16.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:15 smithi067 bash[17385]: cluster 2024-02-10T08:25:14.589825+0000 mgr.y (mgr.14547) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:15 smithi067 bash[24488]: cluster 2024-02-10T08:25:14.589825+0000 mgr.y (mgr.14547) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:17 smithi134 bash[18722]: cluster 2024-02-10T08:25:16.591390+0000 mgr.y (mgr.14547) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:17 smithi067 bash[17385]: cluster 2024-02-10T08:25:16.591390+0000 mgr.y (mgr.14547) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:25:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:17] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-10T08:25:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:17 smithi067 bash[24488]: cluster 2024-02-10T08:25:16.591390+0000 mgr.y (mgr.14547) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:19.363 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:20.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:19 smithi134 bash[18722]: cluster 2024-02-10T08:25:18.592250+0000 mgr.y (mgr.14547) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:19 smithi067 bash[17385]: cluster 2024-02-10T08:25:18.592250+0000 mgr.y (mgr.14547) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:19 smithi067 bash[24488]: cluster 2024-02-10T08:25:18.592250+0000 mgr.y (mgr.14547) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:21.929 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:21 smithi067 bash[17385]: cluster 2024-02-10T08:25:20.592955+0000 mgr.y (mgr.14547) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:21.929 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:21 smithi067 bash[24488]: cluster 2024-02-10T08:25:20.592955+0000 mgr.y (mgr.14547) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:22.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:25:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:25:22.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:21 smithi134 bash[18722]: cluster 2024-02-10T08:25:20.592955+0000 mgr.y (mgr.14547) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/460396045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 msgr2=0x7fc39c1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/460396045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 0x7fc39c1002a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fc398004870 tx=0x7fc398009cb0).stop 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/460396045 shutdown_connections 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/460396045 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 0x7fc39c1002a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/460396045 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc39c0fa620 0x7fc39c0faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/460396045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c100790 0x7fc39c0fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/460396045 >> 172.21.15.67:0/460396045 conn(0x7fc39c0f5ad0 msgr2=0x7fc39c0f7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/460396045 shutdown_connections 2024-02-10T08:25:22.094 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.088+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/460396045 wait complete. 2024-02-10T08:25:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 Processor -- start 2024-02-10T08:25:22.095 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 -- start start 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc39c0fa620 0x7fc39c110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 0x7fc39c110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c100790 0x7fc39c10a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c10a800 con 0x7fc39c0ffec0 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c10a970 con 0x7fc39c100790 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c10ab30 con 0x7fc39c0fa620 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a17d6700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc39c0fa620 0x7fc39c110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 0x7fc39c110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a17d6700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc39c0fa620 0x7fc39c110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:38656/0 (socket says 172.21.15.67:38656) 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a17d6700 1 -- 172.21.15.67:0/732441188 learned_addr learned my addr 172.21.15.67:0/732441188 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:22.096 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a1fd7700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c100790 0x7fc39c10a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:22.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 -- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc39c0fa620 msgr2=0x7fc39c110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:22.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc39c0fa620 0x7fc39c110170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:22.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 -- 172.21.15.67:0/732441188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c100790 msgr2=0x7fc39c10a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:22.097 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc39c100790 0x7fc39c10a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 -- 172.21.15.67:0/732441188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398005040 con 0x7fc39c0ffec0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a0fd5700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 0x7fc39c110660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc38c002730 tx=0x7fc38c009e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c007b40 con 0x7fc39c0ffec0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c10ae10 con 0x7fc39c0ffec0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fc38c00f040 con 0x7fc39c0ffec0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0146d0 con 0x7fc39c0ffec0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.092+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c10b190 con 0x7fc39c0ffec0 2024-02-10T08:25:22.098 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc39c0fe610 con 0x7fc39c0ffec0 2024-02-10T08:25:22.099 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc38c014b00 con 0x7fc39c0ffec0 2024-02-10T08:25:22.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3927fc700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc388061340 0x7fc388063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:22.101 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7fc38c07c970 con 0x7fc39c0ffec0 2024-02-10T08:25:22.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3a17d6700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc388061340 0x7fc388063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:22.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3927fc700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] conn(0x7fc38806b1d0 0x7fc38806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:22.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 --> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc38806dcf0 con 0x7fc38806b1d0 2024-02-10T08:25:22.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc38c081240 con 0x7fc39c0ffec0 2024-02-10T08:25:22.102 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3a1fd7700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] conn(0x7fc38806b1d0 0x7fc38806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:22.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3a17d6700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc388061340 0x7fc388063800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc398004ee0 tx=0x7fc39800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:22.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.096+0000 7fc3a1fd7700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] conn(0x7fc38806b1d0 0x7fc38806d670 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:22.103 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.100+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== osd.3 v2:172.21.15.67:6826/2711595830 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fc38806dcf0 con 0x7fc38806b1d0 2024-02-10T08:25:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 --> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc39c05fab0 con 0x7fc38806b1d0 2024-02-10T08:25:22.120 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3927fc700 1 -- 172.21.15.67:0/732441188 <== osd.3 v2:172.21.15.67:6826/2711595830 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc39c05fab0 con 0x7fc38806b1d0 2024-02-10T08:25:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] conn(0x7fc38806b1d0 msgr2=0x7fc38806d670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6826/2711595830,v1:172.21.15.67:6827/2711595830] conn(0x7fc38806b1d0 0x7fc38806d670 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc388061340 msgr2=0x7fc388063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc388061340 0x7fc388063800 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc398004ee0 tx=0x7fc39800b040).stop 2024-02-10T08:25:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 msgr2=0x7fc39c110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:22.121 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 --2- 172.21.15.67:0/732441188 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc39c0ffec0 0x7fc39c110660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fc38c002730 tx=0x7fc38c009e60).stop 2024-02-10T08:25:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a1fd7700 1 -- 172.21.15.67:0/732441188 reap_dead start 2024-02-10T08:25:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 shutdown_connections 2024-02-10T08:25:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 >> 172.21.15.67:0/732441188 conn(0x7fc39c0f5ad0 msgr2=0x7fc39c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 shutdown_connections 2024-02-10T08:25:22.122 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:22.116+0000 7fc3a3a3a700 1 -- 172.21.15.67:0/732441188 wait complete. 2024-02-10T08:25:22.802 INFO:teuthology.orchestra.run.smithi067.stdout:107374182471 2024-02-10T08:25:22.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-10T08:25:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:25:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:25:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:23 smithi134 bash[18722]: audit 2024-02-10T08:25:21.675505+0000 mgr.y (mgr.14547) 112 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:23 smithi134 bash[18722]: cluster 2024-02-10T08:25:22.594396+0000 mgr.y (mgr.14547) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:23 smithi067 bash[24488]: audit 2024-02-10T08:25:21.675505+0000 mgr.y (mgr.14547) 112 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:23 smithi067 bash[24488]: cluster 2024-02-10T08:25:22.594396+0000 mgr.y (mgr.14547) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:23 smithi067 bash[17385]: audit 2024-02-10T08:25:21.675505+0000 mgr.y (mgr.14547) 112 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:23 smithi067 bash[17385]: cluster 2024-02-10T08:25:22.594396+0000 mgr.y (mgr.14547) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:25 smithi134 bash[18722]: cluster 2024-02-10T08:25:24.595200+0000 mgr.y (mgr.14547) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:25 smithi067 bash[17385]: cluster 2024-02-10T08:25:24.595200+0000 mgr.y (mgr.14547) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:26.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:25 smithi067 bash[24488]: cluster 2024-02-10T08:25:24.595200+0000 mgr.y (mgr.14547) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:27.587 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:27.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:25:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:27] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-10T08:25:27.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:27 smithi067 bash[24488]: cluster 2024-02-10T08:25:26.596559+0000 mgr.y (mgr.14547) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:27.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:27 smithi067 bash[17385]: cluster 2024-02-10T08:25:26.596559+0000 mgr.y (mgr.14547) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:28.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:27 smithi134 bash[18722]: cluster 2024-02-10T08:25:26.596559+0000 mgr.y (mgr.14547) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:29 smithi067 bash[17385]: cluster 2024-02-10T08:25:28.597294+0000 mgr.y (mgr.14547) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:29 smithi067 bash[24488]: cluster 2024-02-10T08:25:28.597294+0000 mgr.y (mgr.14547) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:29 smithi134 bash[18722]: cluster 2024-02-10T08:25:28.597294+0000 mgr.y (mgr.14547) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 -- 172.21.15.67:0/3767887340 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200fa620 msgr2=0x7fce200faa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 --2- 172.21.15.67:0/3767887340 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200fa620 0x7fce200faa80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fce18009be0 tx=0x7fce18004d10).stop 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 -- 172.21.15.67:0/3767887340 shutdown_connections 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 --2- 172.21.15.67:0/3767887340 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce200ffec0 0x7fce201002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 --2- 172.21.15.67:0/3767887340 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200fa620 0x7fce200faa80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 --2- 172.21.15.67:0/3767887340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce20100790 0x7fce200fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 -- 172.21.15.67:0/3767887340 >> 172.21.15.67:0/3767887340 conn(0x7fce200f5ad0 msgr2=0x7fce200f7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.492+0000 7fce25564700 1 -- 172.21.15.67:0/3767887340 shutdown_connections 2024-02-10T08:25:30.499 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 -- 172.21.15.67:0/3767887340 wait complete. 2024-02-10T08:25:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 Processor -- start 2024-02-10T08:25:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 -- start start 2024-02-10T08:25:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce200fa620 0x7fce20110110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:30.500 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200ffec0 0x7fce20110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce20100790 0x7fce2010a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2010a7a0 con 0x7fce20100790 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2010a910 con 0x7fce200fa620 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2010aad0 con 0x7fce200ffec0 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce20100790 0x7fce2010a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200ffec0 0x7fce20110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:30.501 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1f7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce20100790 0x7fce2010a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:53474/0 (socket says 172.21.15.67:53474) 2024-02-10T08:25:30.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1f7fe700 1 -- 172.21.15.67:0/2473143447 learned_addr learned my addr 172.21.15.67:0/2473143447 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:30.502 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1effd700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce200fa620 0x7fce20110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 -- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce200fa620 msgr2=0x7fce20110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce200fa620 0x7fce20110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 -- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce20100790 msgr2=0x7fce2010a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce20100790 0x7fce2010a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 -- 172.21.15.67:0/2473143447 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10005040 con 0x7fce200ffec0 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1f7fe700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fce20100790 0x7fce2010a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce1e7fc700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200ffec0 0x7fce20110600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fce0800e8c0 tx=0x7fce0800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08004620 con 0x7fce200ffec0 2024-02-10T08:25:30.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fce0800f040 con 0x7fce200ffec0 2024-02-10T08:25:30.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.496+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce180097c0 con 0x7fce200ffec0 2024-02-10T08:25:30.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce20116f40 con 0x7fce200ffec0 2024-02-10T08:25:30.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce080098d0 con 0x7fce200ffec0 2024-02-10T08:25:30.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce00000f40 con 0x7fce200ffec0 2024-02-10T08:25:30.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fce08007b40 con 0x7fce200ffec0 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fce0c061340 0x7fce0c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7fce0807c920 con 0x7fce200ffec0 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce1effd700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fce0c061340 0x7fce0c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] conn(0x7fce0c06b060 0x7fce0c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 --> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce0c06db80 con 0x7fce0c06b060 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce0807ccb0 con 0x7fce200ffec0 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce1f7fe700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] conn(0x7fce0c06b060 0x7fce0c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:30.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.500+0000 7fce1effd700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fce0c061340 0x7fce0c063800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fce10009350 tx=0x7fce1000b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:30.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.504+0000 7fce1f7fe700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] conn(0x7fce0c06b060 0x7fce0c06d500 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:30.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.504+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== osd.4 v2:172.21.15.134:6800/636669687 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fce0c06db80 con 0x7fce0c06b060 2024-02-10T08:25:30.528 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 --> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce00002a20 con 0x7fce0c06b060 2024-02-10T08:25:30.529 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce17fff700 1 -- 172.21.15.67:0/2473143447 <== osd.4 v2:172.21.15.134:6800/636669687 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fce00002a20 con 0x7fce0c06b060 2024-02-10T08:25:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] conn(0x7fce0c06b060 msgr2=0x7fce0c06d500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:30.530 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce25564700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.134:6800/636669687,v1:172.21.15.134:6801/636669687] conn(0x7fce0c06b060 0x7fce0c06d500 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fce0c061340 msgr2=0x7fce0c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce25564700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fce0c061340 0x7fce0c063800 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fce10009350 tx=0x7fce1000b040).stop 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.524+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200ffec0 msgr2=0x7fce20110600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.528+0000 7fce25564700 1 --2- 172.21.15.67:0/2473143447 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fce200ffec0 0x7fce20110600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fce0800e8c0 tx=0x7fce0800b040).stop 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.528+0000 7fce1f7fe700 1 -- 172.21.15.67:0/2473143447 reap_dead start 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.528+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 shutdown_connections 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.528+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 >> 172.21.15.67:0/2473143447 conn(0x7fce200f5ad0 msgr2=0x7fce200fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.528+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 shutdown_connections 2024-02-10T08:25:30.531 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:30.528+0000 7fce25564700 1 -- 172.21.15.67:0/2473143447 wait complete. 2024-02-10T08:25:31.309 INFO:teuthology.orchestra.run.smithi067.stdout:128849018943 2024-02-10T08:25:31.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-10T08:25:32.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:25:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:25:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:31 smithi134 bash[18722]: cluster 2024-02-10T08:25:30.598064+0000 mgr.y (mgr.14547) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:31 smithi134 bash[18722]: audit 2024-02-10T08:25:31.361968+0000 mon.c (mon.2) 73 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:25:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:31 smithi134 bash[18722]: audit 2024-02-10T08:25:31.645173+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:31 smithi067 bash[24488]: cluster 2024-02-10T08:25:30.598064+0000 mgr.y (mgr.14547) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:31 smithi067 bash[24488]: audit 2024-02-10T08:25:31.361968+0000 mon.c (mon.2) 73 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:25:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:31 smithi067 bash[24488]: audit 2024-02-10T08:25:31.645173+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:31 smithi067 bash[17385]: cluster 2024-02-10T08:25:30.598064+0000 mgr.y (mgr.14547) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:31 smithi067 bash[17385]: audit 2024-02-10T08:25:31.361968+0000 mon.c (mon.2) 73 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:25:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:31 smithi067 bash[17385]: audit 2024-02-10T08:25:31.645173+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:25:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:25:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:33 smithi134 bash[18722]: audit 2024-02-10T08:25:31.678264+0000 mgr.y (mgr.14547) 118 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:33 smithi134 bash[18722]: cluster 2024-02-10T08:25:32.599479+0000 mgr.y (mgr.14547) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:33 smithi067 bash[17385]: audit 2024-02-10T08:25:31.678264+0000 mgr.y (mgr.14547) 118 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:33 smithi067 bash[17385]: cluster 2024-02-10T08:25:32.599479+0000 mgr.y (mgr.14547) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:33.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:33 smithi067 bash[24488]: audit 2024-02-10T08:25:31.678264+0000 mgr.y (mgr.14547) 118 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:33.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:33 smithi067 bash[24488]: cluster 2024-02-10T08:25:32.599479+0000 mgr.y (mgr.14547) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:35 smithi134 bash[18722]: cluster 2024-02-10T08:25:34.600134+0000 mgr.y (mgr.14547) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:36.099 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:35 smithi067 bash[17385]: cluster 2024-02-10T08:25:34.600134+0000 mgr.y (mgr.14547) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:36.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:35 smithi067 bash[24488]: cluster 2024-02-10T08:25:34.600134+0000 mgr.y (mgr.14547) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[17385]: cluster 2024-02-10T08:25:36.601311+0000 mgr.y (mgr.14547) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[17385]: audit 2024-02-10T08:25:37.384620+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[17385]: audit 2024-02-10T08:25:37.654438+0000 mon.c (mon.2) 74 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:25:37.921 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[17385]: audit 2024-02-10T08:25:37.656109+0000 mon.c (mon.2) 75 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:25:37.922 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:37] "GET /metrics HTTP/1.1" 200 172163 "" "Prometheus/2.33.4" 2024-02-10T08:25:37.922 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[24488]: cluster 2024-02-10T08:25:36.601311+0000 mgr.y (mgr.14547) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:37.922 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[24488]: audit 2024-02-10T08:25:37.384620+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:37.922 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[24488]: audit 2024-02-10T08:25:37.654438+0000 mon.c (mon.2) 74 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:25:37.922 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:37 smithi067 bash[24488]: audit 2024-02-10T08:25:37.656109+0000 mon.c (mon.2) 75 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:25:38.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:37 smithi134 bash[18722]: cluster 2024-02-10T08:25:36.601311+0000 mgr.y (mgr.14547) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:38.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:37 smithi134 bash[18722]: audit 2024-02-10T08:25:37.384620+0000 mon.a (mon.0) 678 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:38.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:37 smithi134 bash[18722]: audit 2024-02-10T08:25:37.654438+0000 mon.c (mon.2) 74 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:25:38.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:37 smithi134 bash[18722]: audit 2024-02-10T08:25:37.656109+0000 mon.c (mon.2) 75 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 -- 172.21.15.67:0/601219598 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f8ed0 msgr2=0x7f7f140f92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 --2- 172.21.15.67:0/601219598 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f8ed0 0x7f7f140f92b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f7f04004870 tx=0x7f7f04009cb0).stop 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 -- 172.21.15.67:0/601219598 shutdown_connections 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 --2- 172.21.15.67:0/601219598 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f14102270 0x7f7f14104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 --2- 172.21.15.67:0/601219598 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f8ed0 0x7f7f140f92b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 --2- 172.21.15.67:0/601219598 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f140f97a0 0x7f7f14101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 -- 172.21.15.67:0/601219598 >> 172.21.15.67:0/601219598 conn(0x7f7f140f27e0 msgr2=0x7f7f140f4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 -- 172.21.15.67:0/601219598 shutdown_connections 2024-02-10T08:25:38.954 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 -- 172.21.15.67:0/601219598 wait complete. 2024-02-10T08:25:38.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 Processor -- start 2024-02-10T08:25:38.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.948+0000 7f7f18864700 1 -- start start 2024-02-10T08:25:38.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f140f8ed0 0x7f7f140f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:38.955 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f97a0 0x7f7f140f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f14102270 0x7f7f140f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14116420 con 0x7f7f140f8ed0 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14116590 con 0x7f7f14102270 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14116750 con 0x7f7f140f97a0 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f97a0 0x7f7f140f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f97a0 0x7f7f140f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:58996/0 (socket says 172.21.15.67:58996) 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 -- 172.21.15.67:0/646721213 learned_addr learned my addr 172.21.15.67:0/646721213 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f1259c700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f140f8ed0 0x7f7f140f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f12d9d700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f14102270 0x7f7f140f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 -- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f14102270 msgr2=0x7f7f140f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f14102270 0x7f7f140f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 -- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f140f8ed0 msgr2=0x7f7f140f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f140f8ed0 0x7f7f140f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 -- 172.21.15.67:0/646721213 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04005040 con 0x7f7f140f97a0 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f1259c700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7f140f8ed0 0x7f7f140f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:25:38.957 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f11d9b700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f97a0 0x7f7f140f71b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f7f0800c7c0 tx=0x7f7f08007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:38.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08004430 con 0x7f7f140f97a0 2024-02-10T08:25:38.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14116a30 con 0x7f7f140f97a0 2024-02-10T08:25:38.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f7f0800e930 con 0x7f7f140f97a0 2024-02-10T08:25:38.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08013960 con 0x7f7f140f97a0 2024-02-10T08:25:38.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14116ff0 con 0x7f7f140f97a0 2024-02-10T08:25:38.958 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.952+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f140f4d40 con 0x7f7f140f97a0 2024-02-10T08:25:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7f0800eaa0 con 0x7f7f140f97a0 2024-02-10T08:25:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f037fe700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7efc061340 0x7f7efc063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f7f0807cc90 con 0x7f7f140f97a0 2024-02-10T08:25:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f1259c700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7efc061340 0x7f7efc063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f037fe700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] conn(0x7f7efc06b250 0x7f7efc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 --> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7efc06dd70 con 0x7f7efc06b250 2024-02-10T08:25:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f0807f1f0 con 0x7f7f140f97a0 2024-02-10T08:25:38.960 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f1259c700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7efc061340 0x7f7efc063800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7f04004ee0 tx=0x7f7f0400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f12d9d700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] conn(0x7f7efc06b250 0x7f7efc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:38.961 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.956+0000 7f7f12d9d700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] conn(0x7f7efc06b250 0x7f7efc06d6f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:38.963 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.960+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== osd.5 v2:172.21.15.134:6808/2103042762 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f7efc06dd70 con 0x7f7efc06b250 2024-02-10T08:25:38.984 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 --> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f140475e0 con 0x7f7efc06b250 2024-02-10T08:25:38.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f037fe700 1 -- 172.21.15.67:0/646721213 <== osd.5 v2:172.21.15.134:6808/2103042762 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7f140475e0 con 0x7f7efc06b250 2024-02-10T08:25:38.985 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] conn(0x7f7efc06b250 msgr2=0x7f7efc06d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.134:6808/2103042762,v1:172.21.15.134:6809/2103042762] conn(0x7f7efc06b250 0x7f7efc06d6f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7efc061340 msgr2=0x7f7efc063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7efc061340 0x7f7efc063800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f7f04004ee0 tx=0x7f7f0400b040).stop 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f97a0 msgr2=0x7f7f140f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 --2- 172.21.15.67:0/646721213 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7f140f97a0 0x7f7f140f71b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f7f0800c7c0 tx=0x7f7f08007ef0).stop 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f12d9d700 1 -- 172.21.15.67:0/646721213 reap_dead start 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 shutdown_connections 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 >> 172.21.15.67:0/646721213 conn(0x7f7f140f27e0 msgr2=0x7f7f140f33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 shutdown_connections 2024-02-10T08:25:38.986 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:38.980+0000 7f7f18864700 1 -- 172.21.15.67:0/646721213 wait complete. 2024-02-10T08:25:39.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:38 smithi134 bash[18722]: audit 2024-02-10T08:25:37.671855+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:39.042 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:38 smithi067 bash[24488]: audit 2024-02-10T08:25:37.671855+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:39.042 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:38 smithi067 bash[17385]: audit 2024-02-10T08:25:37.671855+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:25:39.751 INFO:teuthology.orchestra.run.smithi067.stdout:154618822715 2024-02-10T08:25:39.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-10T08:25:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:39 smithi134 bash[18722]: cluster 2024-02-10T08:25:38.601943+0000 mgr.y (mgr.14547) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:39 smithi067 bash[17385]: cluster 2024-02-10T08:25:38.601943+0000 mgr.y (mgr.14547) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:39 smithi067 bash[24488]: cluster 2024-02-10T08:25:38.601943+0000 mgr.y (mgr.14547) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:40 smithi134 bash[18722]: cluster 2024-02-10T08:25:40.602813+0000 mgr.y (mgr.14547) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:40 smithi067 bash[17385]: cluster 2024-02-10T08:25:40.602813+0000 mgr.y (mgr.14547) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:40 smithi067 bash[24488]: cluster 2024-02-10T08:25:40.602813+0000 mgr.y (mgr.14547) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:42.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:25:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:25:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:43 smithi134 bash[18722]: audit 2024-02-10T08:25:41.689769+0000 mgr.y (mgr.14547) 124 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:43 smithi134 bash[18722]: cluster 2024-02-10T08:25:42.604320+0000 mgr.y (mgr.14547) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:43.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:25:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:25:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:43 smithi067 bash[24488]: audit 2024-02-10T08:25:41.689769+0000 mgr.y (mgr.14547) 124 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:43 smithi067 bash[24488]: cluster 2024-02-10T08:25:42.604320+0000 mgr.y (mgr.14547) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:43 smithi067 bash[17385]: audit 2024-02-10T08:25:41.689769+0000 mgr.y (mgr.14547) 124 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:43 smithi067 bash[17385]: cluster 2024-02-10T08:25:42.604320+0000 mgr.y (mgr.14547) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:44.535 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:45.809 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:45 smithi067 bash[17385]: cluster 2024-02-10T08:25:44.604921+0000 mgr.y (mgr.14547) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:45.809 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:45 smithi067 bash[24488]: cluster 2024-02-10T08:25:44.604921+0000 mgr.y (mgr.14547) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:46.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:45 smithi134 bash[18722]: cluster 2024-02-10T08:25:44.604921+0000 mgr.y (mgr.14547) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:47.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/4187339130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f90100790 msgr2=0x7f8f900fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:47.191 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- 172.21.15.67:0/4187339130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f90100790 0x7f8f900fa050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f8f80004870 tx=0x7f8f80009cb0).stop 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/4187339130 shutdown_connections 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- 172.21.15.67:0/4187339130 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 0x7f8f900faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- 172.21.15.67:0/4187339130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f900ffec0 0x7f8f901002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- 172.21.15.67:0/4187339130 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f90100790 0x7f8f900fa050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/4187339130 >> 172.21.15.67:0/4187339130 conn(0x7f8f900f5ad0 msgr2=0x7f8f900f7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/4187339130 shutdown_connections 2024-02-10T08:25:47.192 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/4187339130 wait complete. 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 Processor -- start 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- start start 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 0x7f8f90110200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f900ffec0 0x7f8f901106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f90100790 0x7f8f9010a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9010a890 con 0x7f8f900ffec0 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9010aa00 con 0x7f8f90100790 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9010abc0 con 0x7f8f900fa620 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 0x7f8f90110200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 0x7f8f90110200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:46876/0 (socket says 172.21.15.67:46876) 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 -- 172.21.15.67:0/2569902623 learned_addr learned my addr 172.21.15.67:0/2569902623 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:47.193 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f94d39700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f900ffec0 0x7f8f901106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:47.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 -- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f90100790 msgr2=0x7f8f9010a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:47.194 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f90100790 0x7f8f9010a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 -- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f900ffec0 msgr2=0x7f8f901106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f900ffec0 0x7f8f901106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 -- 172.21.15.67:0/2569902623 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80005040 con 0x7f8f900fa620 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f94d39700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8f900ffec0 0x7f8f901106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9553a700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 0x7f8f90110200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8f8c00c7c0 tx=0x7f8f8c007ef0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c004430 con 0x7f8f900fa620 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f8f8c00a990 con 0x7f8f900fa620 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9010aea0 con 0x7f8f900fa620 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c013990 con 0x7f8f900fa620 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.188+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9010b220 con 0x7f8f900fa620 2024-02-10T08:25:47.195 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f900fe630 con 0x7f8f900fa620 2024-02-10T08:25:47.196 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8f8c00ab00 con 0x7f8f900fa620 2024-02-10T08:25:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f867fc700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8f7c061340 0x7f8f7c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f8f8c07dd20 con 0x7f8f900fa620 2024-02-10T08:25:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f94d39700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8f7c061340 0x7f8f7c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f867fc700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] conn(0x7f8f7c06b250 0x7f8f7c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 --> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f7c06dd70 con 0x7f8f7c06b250 2024-02-10T08:25:47.197 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f8c07e100 con 0x7f8f900fa620 2024-02-10T08:25:47.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f94d39700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8f7c061340 0x7f8f7c063800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f8f80004ee0 tx=0x7f8f80009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:47.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.192+0000 7f8f95d3b700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] conn(0x7f8f7c06b250 0x7f8f7c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:47.198 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.196+0000 7f8f95d3b700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] conn(0x7f8f7c06b250 0x7f8f7c06d6f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:47.200 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.196+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== osd.6 v2:172.21.15.134:6816/3982264053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8f7c06dd70 con 0x7f8f7c06b250 2024-02-10T08:25:47.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 --> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f9005fab0 con 0x7f8f7c06b250 2024-02-10T08:25:47.220 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f867fc700 1 -- 172.21.15.67:0/2569902623 <== osd.6 v2:172.21.15.134:6816/3982264053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8f9005fab0 con 0x7f8f7c06b250 2024-02-10T08:25:47.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] conn(0x7f8f7c06b250 msgr2=0x7f8f7c06d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:47.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.134:6816/3982264053,v1:172.21.15.134:6817/3982264053] conn(0x7f8f7c06b250 0x7f8f7c06d6f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:47.221 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8f7c061340 msgr2=0x7f8f7c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8f7c061340 0x7f8f7c063800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f8f80004ee0 tx=0x7f8f80009cb0).stop 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 msgr2=0x7f8f90110200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 --2- 172.21.15.67:0/2569902623 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8f900fa620 0x7f8f90110200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8f8c00c7c0 tx=0x7f8f8c007ef0).stop 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f95d3b700 1 -- 172.21.15.67:0/2569902623 reap_dead start 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 shutdown_connections 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 >> 172.21.15.67:0/2569902623 conn(0x7f8f900f5ad0 msgr2=0x7f8f900fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 shutdown_connections 2024-02-10T08:25:47.222 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:47.216+0000 7f8f9779e700 1 -- 172.21.15.67:0/2569902623 wait complete. 2024-02-10T08:25:47.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:47 smithi067 bash[17385]: cluster 2024-02-10T08:25:46.606258+0000 mgr.y (mgr.14547) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:47.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:47 smithi067 bash[24488]: cluster 2024-02-10T08:25:46.606258+0000 mgr.y (mgr.14547) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:47.998 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:25:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:47] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-10T08:25:47.998 INFO:teuthology.orchestra.run.smithi067.stdout:180388626484 2024-02-10T08:25:47.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-10T08:25:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:47 smithi134 bash[18722]: cluster 2024-02-10T08:25:46.606258+0000 mgr.y (mgr.14547) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:49 smithi134 bash[18722]: cluster 2024-02-10T08:25:48.607071+0000 mgr.y (mgr.14547) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:49 smithi067 bash[17385]: cluster 2024-02-10T08:25:48.607071+0000 mgr.y (mgr.14547) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:49 smithi067 bash[24488]: cluster 2024-02-10T08:25:48.607071+0000 mgr.y (mgr.14547) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:52.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:51 smithi134 bash[18722]: cluster 2024-02-10T08:25:50.607780+0000 mgr.y (mgr.14547) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:52.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:25:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:25:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:51 smithi067 bash[17385]: cluster 2024-02-10T08:25:50.607780+0000 mgr.y (mgr.14547) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:51 smithi067 bash[24488]: cluster 2024-02-10T08:25:50.607780+0000 mgr.y (mgr.14547) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:52.786 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:25:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:53 smithi134 bash[18722]: audit 2024-02-10T08:25:51.699559+0000 mgr.y (mgr.14547) 130 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:53 smithi134 bash[18722]: cluster 2024-02-10T08:25:52.609243+0000 mgr.y (mgr.14547) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:53.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:25:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:25:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:53 smithi067 bash[17385]: audit 2024-02-10T08:25:51.699559+0000 mgr.y (mgr.14547) 130 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:53 smithi067 bash[17385]: cluster 2024-02-10T08:25:52.609243+0000 mgr.y (mgr.14547) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:53 smithi067 bash[24488]: audit 2024-02-10T08:25:51.699559+0000 mgr.y (mgr.14547) 130 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:25:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:53 smithi067 bash[24488]: cluster 2024-02-10T08:25:52.609243+0000 mgr.y (mgr.14547) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 -- 172.21.15.67:0/352535967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 msgr2=0x7f023c104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 --2- 172.21.15.67:0/352535967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c104b60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f0234009c30 tx=0x7f0234004c50).stop 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 -- 172.21.15.67:0/352535967 shutdown_connections 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 --2- 172.21.15.67:0/352535967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c104b60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 --2- 172.21.15.67:0/352535967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f023c0fdd60 0x7f023c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 --2- 172.21.15.67:0/352535967 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f023c0fe6c0 0x7f023c0f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 -- 172.21.15.67:0/352535967 >> 172.21.15.67:0/352535967 conn(0x7f023c0701a0 msgr2=0x7f023c0f44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 -- 172.21.15.67:0/352535967 shutdown_connections 2024-02-10T08:25:55.574 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.568+0000 7f0242d46700 1 -- 172.21.15.67:0/352535967 wait complete. 2024-02-10T08:25:55.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 Processor -- start 2024-02-10T08:25:55.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 -- start start 2024-02-10T08:25:55.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:55.575 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f023c0fdd60 0x7f023c10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f023c0fe6c0 0x7f023c108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:55.576 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c108680 con 0x7f023c0f6850 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c1087f0 con 0x7f023c0fdd60 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023c1089b0 con 0x7f023c0fe6c0 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c10dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:54206/0 (socket says 172.21.15.67:54206) 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 -- 172.21.15.67:0/2522793528 learned_addr learned my addr 172.21.15.67:0/2522793528 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f02412e3700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f023c0fe6c0 0x7f023c108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f023bfff700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f023c0fdd60 0x7f023c10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 -- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f023c0fe6c0 msgr2=0x7f023c108190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f023c0fe6c0 0x7f023c108190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 -- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f023c0fdd60 msgr2=0x7f023c10e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f023c0fdd60 0x7f023c10e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:55.577 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 -- 172.21.15.67:0/2522793528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228005040 con 0x7f023c0f6850 2024-02-10T08:25:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f02412e3700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f023c0fe6c0 0x7f023c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:25:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0240ae2700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c10dff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f023000e970 tx=0x7f023000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230004660 con 0x7f023c0f6850 2024-02-10T08:25:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f023000f040 con 0x7f023c0f6850 2024-02-10T08:25:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02300097f0 con 0x7f023c0f6850 2024-02-10T08:25:55.578 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02340097c0 con 0x7f023c0f6850 2024-02-10T08:25:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.572+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c108f40 con 0x7f023c0f6850 2024-02-10T08:25:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f023c06cec0 con 0x7f023c0f6850 2024-02-10T08:25:55.579 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f0230007b40 con 0x7f023c0f6850 2024-02-10T08:25:55.580 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0239ffb700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f022c061340 0x7f022c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f023007c920 con 0x7f023c0f6850 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f023bfff700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f022c061340 0x7f022c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0239ffb700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] conn(0x7f022c06b250 0x7f022c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 --> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f022c06dd70 con 0x7f022c06b250 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0230081220 con 0x7f023c0f6850 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f023bfff700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f022c061340 0x7f022c063800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f023c109930 tx=0x7f022800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:55.581 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f02412e3700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] conn(0x7f022c06b250 0x7f022c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:25:55.582 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.576+0000 7f02412e3700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] conn(0x7f022c06b250 0x7f022c06d6f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:25:55.583 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.580+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== osd.7 v2:172.21.15.134:6824/1760805884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f022c06dd70 con 0x7f022c06b250 2024-02-10T08:25:55.604 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 --> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f023c05fad0 con 0x7f022c06b250 2024-02-10T08:25:55.605 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0239ffb700 1 -- 172.21.15.67:0/2522793528 <== osd.7 v2:172.21.15.134:6824/1760805884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f023c05fad0 con 0x7f022c06b250 2024-02-10T08:25:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] conn(0x7f022c06b250 msgr2=0x7f022c06d6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.134:6824/1760805884,v1:172.21.15.134:6825/1760805884] conn(0x7f022c06b250 0x7f022c06d6f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:25:55.606 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f022c061340 msgr2=0x7f022c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f022c061340 0x7f022c063800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f023c109930 tx=0x7f022800b040).stop 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 msgr2=0x7f023c10dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 --2- 172.21.15.67:0/2522793528 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f023c0f6850 0x7f023c10dff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f023000e970 tx=0x7f023000b040).stop 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f02412e3700 1 -- 172.21.15.67:0/2522793528 reap_dead start 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 shutdown_connections 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.600+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 >> 172.21.15.67:0/2522793528 conn(0x7f023c0701a0 msgr2=0x7f023c06c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.604+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 shutdown_connections 2024-02-10T08:25:55.607 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:25:55.604+0000 7f0242d46700 1 -- 172.21.15.67:0/2522793528 wait complete. 2024-02-10T08:25:56.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:55 smithi134 bash[18722]: cluster 2024-02-10T08:25:54.610032+0000 mgr.y (mgr.14547) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:55 smithi067 bash[17385]: cluster 2024-02-10T08:25:54.610032+0000 mgr.y (mgr.14547) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:55 smithi067 bash[24488]: cluster 2024-02-10T08:25:54.610032+0000 mgr.y (mgr.14547) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:25:56.429 INFO:teuthology.orchestra.run.smithi067.stdout:206158430253 2024-02-10T08:25:56.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-10T08:25:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:57 smithi134 bash[18722]: cluster 2024-02-10T08:25:56.611433+0000 mgr.y (mgr.14547) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:57 smithi067 bash[17385]: cluster 2024-02-10T08:25:56.611433+0000 mgr.y (mgr.14547) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:25:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:25:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:25:57] "GET /metrics HTTP/1.1" 200 172162 "" "Prometheus/2.33.4" 2024-02-10T08:25:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:57 smithi067 bash[24488]: cluster 2024-02-10T08:25:56.611433+0000 mgr.y (mgr.14547) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:00.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:25:59 smithi134 bash[18722]: cluster 2024-02-10T08:25:58.612229+0000 mgr.y (mgr.14547) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:00.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:25:59 smithi067 bash[17385]: cluster 2024-02-10T08:25:58.612229+0000 mgr.y (mgr.14547) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:00.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:25:59 smithi067 bash[24488]: cluster 2024-02-10T08:25:58.612229+0000 mgr.y (mgr.14547) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:01.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:00 smithi134 bash[18722]: cluster 2024-02-10T08:26:00.613054+0000 mgr.y (mgr.14547) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:00 smithi067 bash[17385]: cluster 2024-02-10T08:26:00.613054+0000 mgr.y (mgr.14547) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:01.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:00 smithi067 bash[24488]: cluster 2024-02-10T08:26:00.613054+0000 mgr.y (mgr.14547) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:01.208 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:02.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:26:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:26:03.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:03 smithi067 bash[17385]: audit 2024-02-10T08:26:01.711017+0000 mgr.y (mgr.14547) 136 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:03.175 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:03 smithi067 bash[17385]: cluster 2024-02-10T08:26:02.614616+0000 mgr.y (mgr.14547) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:03.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:26:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:26:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:03 smithi134 bash[18722]: audit 2024-02-10T08:26:01.711017+0000 mgr.y (mgr.14547) 136 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:03 smithi134 bash[18722]: cluster 2024-02-10T08:26:02.614616+0000 mgr.y (mgr.14547) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:03.618 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:03 smithi067 bash[24488]: audit 2024-02-10T08:26:01.711017+0000 mgr.y (mgr.14547) 136 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:03.618 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:03 smithi067 bash[24488]: cluster 2024-02-10T08:26:02.614616+0000 mgr.y (mgr.14547) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:03.785 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 -- 172.21.15.67:0/1188951860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f8c70 msgr2=0x7f381c0f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 --2- 172.21.15.67:0/1188951860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f8c70 0x7f381c0f9050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f380c004870 tx=0x7f380c009cb0).stop 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 -- 172.21.15.67:0/1188951860 shutdown_connections 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 --2- 172.21.15.67:0/1188951860 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f8c70 0x7f381c0f9050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 --2- 172.21.15.67:0/1188951860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f381c1021d0 0x7f381c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 --2- 172.21.15.67:0/1188951860 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c0f95d0 0x7f381c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 -- 172.21.15.67:0/1188951860 >> 172.21.15.67:0/1188951860 conn(0x7f381c0f27a0 msgr2=0x7f381c0f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 -- 172.21.15.67:0/1188951860 shutdown_connections 2024-02-10T08:26:03.786 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.779+0000 7f3821c19700 1 -- 172.21.15.67:0/1188951860 wait complete. 2024-02-10T08:26:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 Processor -- start 2024-02-10T08:26:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- start start 2024-02-10T08:26:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f381c0f8c70 0x7f381c10e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 0x7f381c10e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c1021d0 0x7f381c108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:03.787 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1086f0 con 0x7f381c0f95d0 2024-02-10T08:26:03.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c108860 con 0x7f381c0f8c70 2024-02-10T08:26:03.788 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c108a20 con 0x7f381c1021d0 2024-02-10T08:26:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 0x7f381c10e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f381bfff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c1021d0 0x7f381c108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 0x7f381c10e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:59886/0 (socket says 172.21.15.67:59886) 2024-02-10T08:26:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 -- 172.21.15.67:0/1061025964 learned_addr learned my addr 172.21.15.67:0/1061025964 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f381b7fe700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f381c0f8c70 0x7f381c10e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:03.789 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 -- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c1021d0 msgr2=0x7f381c108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c1021d0 0x7f381c108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 -- 172.21.15.67:0/1061025964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f381c0f8c70 msgr2=0x7f381c10e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f381c0f8c70 0x7f381c10e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 -- 172.21.15.67:0/1061025964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c005040 con 0x7f381c0f95d0 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f381bfff700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c1021d0 0x7f381c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3813fff700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 0x7f381c10e550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f380400e970 tx=0x7f380400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804004660 con 0x7f381c0f95d0 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f380400f040 con 0x7f381c0f95d0 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c108d00 con 0x7f381c0f95d0 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040146f0 con 0x7f381c0f95d0 2024-02-10T08:26:03.790 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c109080 con 0x7f381c0f95d0 2024-02-10T08:26:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.783+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c0620e0 con 0x7f381c0f95d0 2024-02-10T08:26:03.791 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.787+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3804007b40 con 0x7f381c0f95d0 2024-02-10T08:26:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.787+0000 7f38197fa700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3808061340 0x7f3808063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.787+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f380407c940 con 0x7f381c0f95d0 2024-02-10T08:26:03.792 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.787+0000 7f381b7fe700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3808061340 0x7f3808063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:03.793 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.787+0000 7f381b7fe700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3808061340 0x7f3808063800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f380c005010 tx=0x7f380c00b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:03.799 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.795+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f380404c830 con 0x7f381c0f95d0 2024-02-10T08:26:03.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.987+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f381c0475e0 con 0x7f381c0f95d0 2024-02-10T08:26:03.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.987+0000 7f38197fa700 1 -- 172.21.15.67:0/1061025964 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3804019090 con 0x7f381c0f95d0 2024-02-10T08:26:03.992 INFO:teuthology.orchestra.run.smithi067.stdout:34359738475 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3808061340 msgr2=0x7f3808063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3808061340 0x7f3808063800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f380c005010 tx=0x7f380c00b040).stop 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 msgr2=0x7f381c10e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 0x7f381c10e550 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f380400e970 tx=0x7f380400b040).stop 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 shutdown_connections 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3808061340 0x7f3808063800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f381c1021d0 0x7f381c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f381c0f8c70 0x7f381c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 --2- 172.21.15.67:0/1061025964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f381c0f95d0 0x7f381c10e550 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 >> 172.21.15.67:0/1061025964 conn(0x7f381c0f27a0 msgr2=0x7f381c0f33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:03.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 shutdown_connections 2024-02-10T08:26:03.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:03.991+0000 7f3821c19700 1 -- 172.21.15.67:0/1061025964 wait complete. 2024-02-10T08:26:04.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:04 smithi067 bash[17385]: audit 2024-02-10T08:26:03.991942+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.67:0/1061025964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T08:26:04.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:04 smithi067 bash[24488]: audit 2024-02-10T08:26:03.991942+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.67:0/1061025964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T08:26:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:04 smithi134 bash[18722]: audit 2024-02-10T08:26:03.991942+0000 mon.a (mon.0) 680 : audit [DBG] from='client.? 172.21.15.67:0/1061025964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T08:26:04.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738462 got 34359738475 for osd.0 2024-02-10T08:26:04.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-10T08:26:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:05 smithi134 bash[18722]: cluster 2024-02-10T08:26:04.615391+0000 mgr.y (mgr.14547) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:05 smithi067 bash[17385]: cluster 2024-02-10T08:26:04.615391+0000 mgr.y (mgr.14547) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:05 smithi067 bash[24488]: cluster 2024-02-10T08:26:04.615391+0000 mgr.y (mgr.14547) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:07 smithi134 bash[18722]: cluster 2024-02-10T08:26:06.616893+0000 mgr.y (mgr.14547) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:07 smithi067 bash[17385]: cluster 2024-02-10T08:26:06.616893+0000 mgr.y (mgr.14547) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:26:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:07] "GET /metrics HTTP/1.1" 200 172166 "" "Prometheus/2.33.4" 2024-02-10T08:26:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:07 smithi067 bash[24488]: cluster 2024-02-10T08:26:06.616893+0000 mgr.y (mgr.14547) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:09.569 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:09 smithi067 bash[17385]: cluster 2024-02-10T08:26:08.617739+0000 mgr.y (mgr.14547) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:09.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:09 smithi067 bash[24488]: cluster 2024-02-10T08:26:08.617739+0000 mgr.y (mgr.14547) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:09 smithi134 bash[18722]: cluster 2024-02-10T08:26:08.617739+0000 mgr.y (mgr.14547) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:10 smithi067 bash[17385]: cluster 2024-02-10T08:26:10.618535+0000 mgr.y (mgr.14547) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:10 smithi067 bash[24488]: cluster 2024-02-10T08:26:10.618535+0000 mgr.y (mgr.14547) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:11.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:10 smithi134 bash[18722]: cluster 2024-02-10T08:26:10.618535+0000 mgr.y (mgr.14547) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:12.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:26:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:26:12.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2871269189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fdd10 msgr2=0x7f8aa80fe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:12.453 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 --2- 172.21.15.67:0/2871269189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fdd10 0x7f8aa80fe0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f8a9c004870 tx=0x7f8a9c009cb0).stop 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2871269189 shutdown_connections 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 --2- 172.21.15.67:0/2871269189 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fdd10 0x7f8aa80fe0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 --2- 172.21.15.67:0/2871269189 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80f6800 0x7f8aa8104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 --2- 172.21.15.67:0/2871269189 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa80fe670 0x7f8aa80f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2871269189 >> 172.21.15.67:0/2871269189 conn(0x7f8aa80700c0 msgr2=0x7f8aa80f4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2871269189 shutdown_connections 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.447+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2871269189 wait complete. 2024-02-10T08:26:12.454 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 Processor -- start 2024-02-10T08:26:12.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 -- start start 2024-02-10T08:26:12.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa80f6800 0x7f8aa810e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:12.455 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 0x7f8aa810e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:12.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fe670 0x7f8aa8108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:12.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa81086f0 con 0x7f8aa80fe670 2024-02-10T08:26:12.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8108860 con 0x7f8aa80f6800 2024-02-10T08:26:12.456 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8108a20 con 0x7f8aa80fdd10 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa5d9b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 0x7f8aa810e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fe670 0x7f8aa8108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa5d9b700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 0x7f8aa810e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:36964/0 (socket says 172.21.15.67:36964) 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa5d9b700 1 -- 172.21.15.67:0/2902019664 learned_addr learned my addr 172.21.15.67:0/2902019664 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa659c700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa80f6800 0x7f8aa810e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 -- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 msgr2=0x7f8aa810e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 0x7f8aa810e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 -- 172.21.15.67:0/2902019664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa80f6800 msgr2=0x7f8aa810e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:12.457 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa80f6800 0x7f8aa810e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 -- 172.21.15.67:0/2902019664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00097c0 con 0x7f8aa80fe670 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa5d9b700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 0x7f8aa810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aa6d9d700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fe670 0x7f8aa8108200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f8aa0001cb0 tx=0x7f8aa0004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00079b0 con 0x7f8aa80fe670 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2902019664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c005040 con 0x7f8aa80fe670 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f8aa00090d0 con 0x7f8aa80fe670 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000fb40 con 0x7f8aa80fe670 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8aaccb6700 1 -- 172.21.15.67:0/2902019664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa8108f80 con 0x7f8aa80fe670 2024-02-10T08:26:12.458 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.451+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c004f20 con 0x7f8aa80fe670 2024-02-10T08:26:12.459 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.455+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8aa001f440 con 0x7f8aa80fe670 2024-02-10T08:26:12.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.455+0000 7f8a977fe700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a90061340 0x7f8a90063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:12.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.455+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f8aa007d770 con 0x7f8aa80fe670 2024-02-10T08:26:12.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.455+0000 7f8aa659c700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a90061340 0x7f8a90063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:12.460 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.455+0000 7f8aa659c700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a90061340 0x7f8a90063800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8a9c005010 tx=0x7f8a9c00b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:12.465 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.459+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8aa004d430 con 0x7f8aa80fe670 2024-02-10T08:26:12.649 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.643+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8a8c0051d0 con 0x7f8aa80fe670 2024-02-10T08:26:12.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.643+0000 7f8a977fe700 1 -- 172.21.15.67:0/2902019664 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8aa0050a50 con 0x7f8aa80fe670 2024-02-10T08:26:12.650 INFO:teuthology.orchestra.run.smithi067.stdout:55834574947 2024-02-10T08:26:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a90061340 msgr2=0x7f8a90063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a90061340 0x7f8a90063800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f8a9c005010 tx=0x7f8a9c00b040).stop 2024-02-10T08:26:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fe670 msgr2=0x7f8aa8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fe670 0x7f8aa8108200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f8aa0001cb0 tx=0x7f8aa0004c50).stop 2024-02-10T08:26:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 shutdown_connections 2024-02-10T08:26:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa80fe670 0x7f8aa8108200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a90061340 0x7f8a90063800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa80f6800 0x7f8aa810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 --2- 172.21.15.67:0/2902019664 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa80fdd10 0x7f8aa810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:12.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 >> 172.21.15.67:0/2902019664 conn(0x7f8aa80700c0 msgr2=0x7f8aa806c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:12.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 shutdown_connections 2024-02-10T08:26:12.653 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:12.647+0000 7f8a957fa700 1 -- 172.21.15.67:0/2902019664 wait complete. 2024-02-10T08:26:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:13 smithi067 bash[17385]: audit 2024-02-10T08:26:11.722441+0000 mgr.y (mgr.14547) 142 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:13 smithi067 bash[17385]: cluster 2024-02-10T08:26:12.619923+0000 mgr.y (mgr.14547) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:13 smithi067 bash[17385]: audit 2024-02-10T08:26:12.649712+0000 mon.a (mon.0) 681 : audit [DBG] from='client.? 172.21.15.67:0/2902019664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T08:26:13.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:13 smithi067 bash[24488]: audit 2024-02-10T08:26:11.722441+0000 mgr.y (mgr.14547) 142 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:13 smithi067 bash[24488]: cluster 2024-02-10T08:26:12.619923+0000 mgr.y (mgr.14547) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:13 smithi067 bash[24488]: audit 2024-02-10T08:26:12.649712+0000 mon.a (mon.0) 681 : audit [DBG] from='client.? 172.21.15.67:0/2902019664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T08:26:13.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574934 got 55834574947 for osd.1 2024-02-10T08:26:13.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-10T08:26:13.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:26:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:26:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:13 smithi134 bash[18722]: audit 2024-02-10T08:26:11.722441+0000 mgr.y (mgr.14547) 142 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:13 smithi134 bash[18722]: cluster 2024-02-10T08:26:12.619923+0000 mgr.y (mgr.14547) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:13 smithi134 bash[18722]: audit 2024-02-10T08:26:12.649712+0000 mon.a (mon.0) 681 : audit [DBG] from='client.? 172.21.15.67:0/2902019664' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T08:26:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:15 smithi134 bash[18722]: cluster 2024-02-10T08:26:14.620611+0000 mgr.y (mgr.14547) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:16.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:15 smithi067 bash[17385]: cluster 2024-02-10T08:26:14.620611+0000 mgr.y (mgr.14547) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:15 smithi067 bash[24488]: cluster 2024-02-10T08:26:14.620611+0000 mgr.y (mgr.14547) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:17 smithi134 bash[18722]: cluster 2024-02-10T08:26:16.622018+0000 mgr.y (mgr.14547) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:17 smithi067 bash[17385]: cluster 2024-02-10T08:26:16.622018+0000 mgr.y (mgr.14547) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:26:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:17] "GET /metrics HTTP/1.1" 200 172166 "" "Prometheus/2.33.4" 2024-02-10T08:26:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:17 smithi067 bash[24488]: cluster 2024-02-10T08:26:16.622018+0000 mgr.y (mgr.14547) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:18.195 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:18 smithi134 bash[18722]: cluster 2024-02-10T08:26:18.622818+0000 mgr.y (mgr.14547) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:19.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:18 smithi067 bash[17385]: cluster 2024-02-10T08:26:18.622818+0000 mgr.y (mgr.14547) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:19.057 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:18 smithi067 bash[24488]: cluster 2024-02-10T08:26:18.622818+0000 mgr.y (mgr.14547) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:20.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 -- 172.21.15.67:0/527265491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600fcb70 msgr2=0x7f3a600fcf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 --2- 172.21.15.67:0/527265491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600fcb70 0x7f3a600fcf50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3a50009be0 tx=0x7f3a50004d10).stop 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 -- 172.21.15.67:0/527265491 shutdown_connections 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 --2- 172.21.15.67:0/527265491 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600fcb70 0x7f3a600fcf50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 --2- 172.21.15.67:0/527265491 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600f73e0 0x7f3a601039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 --2- 172.21.15.67:0/527265491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a600fd4d0 0x7f3a600f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 -- 172.21.15.67:0/527265491 >> 172.21.15.67:0/527265491 conn(0x7f3a600f27a0 msgr2=0x7f3a600f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 -- 172.21.15.67:0/527265491 shutdown_connections 2024-02-10T08:26:20.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 -- 172.21.15.67:0/527265491 wait complete. 2024-02-10T08:26:20.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.819+0000 7f3a66913700 1 Processor -- start 2024-02-10T08:26:20.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 -- start start 2024-02-10T08:26:20.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 0x7f3a6010ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:20.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a600fcb70 0x7f3a6010d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600fd4d0 0x7f3a60106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60107490 con 0x7f3a600f73e0 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60107600 con 0x7f3a600fcb70 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601077c0 con 0x7f3a600fd4d0 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 0x7f3a6010ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5ffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 0x7f3a6010ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:42916/0 (socket says 172.21.15.67:42916) 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5ffff700 1 -- 172.21.15.67:0/1354853634 learned_addr learned my addr 172.21.15.67:0/1354853634 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600fd4d0 0x7f3a60106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:20.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5f7fe700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a600fcb70 0x7f3a6010d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:20.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 -- 172.21.15.67:0/1354853634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a600fcb70 msgr2=0x7f3a6010d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:20.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a600fcb70 0x7f3a6010d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 -- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 msgr2=0x7f3a6010ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 0x7f3a6010ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 -- 172.21.15.67:0/1354853634 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500097c0 con 0x7f3a600fd4d0 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5ffff700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 0x7f3a6010ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a64eb0700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600fd4d0 0x7f3a60106fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f3a58010a80 tx=0x7f3a5800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58010ec0 con 0x7f3a600fd4d0 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a60107a40 con 0x7f3a600fd4d0 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f3a5800e760 con 0x7f3a600fd4d0 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a60060ed0 con 0x7f3a600fd4d0 2024-02-10T08:26:20.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.823+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58009340 con 0x7f3a600fd4d0 2024-02-10T08:26:20.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.827+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60062100 con 0x7f3a600fd4d0 2024-02-10T08:26:20.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.827+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f3a58009770 con 0x7f3a600fd4d0 2024-02-10T08:26:20.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.827+0000 7f3a5d7fa700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3a4c061340 0x7f3a4c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:20.832 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.827+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f3a5807eee0 con 0x7f3a600fd4d0 2024-02-10T08:26:20.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.827+0000 7f3a5ffff700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3a4c061340 0x7f3a4c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:20.833 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.827+0000 7f3a5ffff700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3a4c061340 0x7f3a4c063800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3a50004e00 tx=0x7f3a50004d10).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:20.840 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:20.831+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f3a5804edb0 con 0x7f3a600fd4d0 2024-02-10T08:26:21.023 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.019+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a60105ea0 con 0x7f3a600fd4d0 2024-02-10T08:26:21.024 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.019+0000 7f3a5d7fa700 1 -- 172.21.15.67:0/1354853634 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a5801f030 con 0x7f3a600fd4d0 2024-02-10T08:26:21.024 INFO:teuthology.orchestra.run.smithi067.stdout:77309411419 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3a4c061340 msgr2=0x7f3a4c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3a4c061340 0x7f3a4c063800 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f3a50004e00 tx=0x7f3a50004d10).stop 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600fd4d0 msgr2=0x7f3a60106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600fd4d0 0x7f3a60106fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f3a58010a80 tx=0x7f3a5800b040).stop 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 shutdown_connections 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f3a600f73e0 0x7f3a6010ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f3a4c061340 0x7f3a4c063800 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f3a600fd4d0 0x7f3a60106fa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 --2- 172.21.15.67:0/1354853634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a600fcb70 0x7f3a6010d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 >> 172.21.15.67:0/1354853634 conn(0x7f3a600f27a0 msgr2=0x7f3a600f4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 shutdown_connections 2024-02-10T08:26:21.029 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:21.023+0000 7f3a66913700 1 -- 172.21.15.67:0/1354853634 wait complete. 2024-02-10T08:26:21.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411419 for osd.2 2024-02-10T08:26:21.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-10T08:26:22.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:26:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:26:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:21 smithi134 bash[18722]: cluster 2024-02-10T08:26:20.623457+0000 mgr.y (mgr.14547) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:21 smithi134 bash[18722]: audit 2024-02-10T08:26:21.023855+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.67:0/1354853634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T08:26:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:21 smithi067 bash[17385]: cluster 2024-02-10T08:26:20.623457+0000 mgr.y (mgr.14547) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:21 smithi067 bash[17385]: audit 2024-02-10T08:26:21.023855+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.67:0/1354853634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T08:26:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:21 smithi067 bash[24488]: cluster 2024-02-10T08:26:20.623457+0000 mgr.y (mgr.14547) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:21 smithi067 bash[24488]: audit 2024-02-10T08:26:21.023855+0000 mon.c (mon.2) 76 : audit [DBG] from='client.? 172.21.15.67:0/1354853634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T08:26:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:26:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:26:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:23 smithi134 bash[18722]: audit 2024-02-10T08:26:21.733912+0000 mgr.y (mgr.14547) 148 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:23 smithi134 bash[18722]: cluster 2024-02-10T08:26:22.624880+0000 mgr.y (mgr.14547) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:23 smithi067 bash[17385]: audit 2024-02-10T08:26:21.733912+0000 mgr.y (mgr.14547) 148 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:23 smithi067 bash[17385]: cluster 2024-02-10T08:26:22.624880+0000 mgr.y (mgr.14547) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:23 smithi067 bash[24488]: audit 2024-02-10T08:26:21.733912+0000 mgr.y (mgr.14547) 148 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:23 smithi067 bash[24488]: cluster 2024-02-10T08:26:22.624880+0000 mgr.y (mgr.14547) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:25 smithi134 bash[18722]: cluster 2024-02-10T08:26:24.625582+0000 mgr.y (mgr.14547) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:25 smithi067 bash[17385]: cluster 2024-02-10T08:26:24.625582+0000 mgr.y (mgr.14547) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:26.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:25 smithi067 bash[24488]: cluster 2024-02-10T08:26:24.625582+0000 mgr.y (mgr.14547) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:26.588 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:26.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:26 smithi067 bash[17385]: cluster 2024-02-10T08:26:26.626903+0000 mgr.y (mgr.14547) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:26.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:26 smithi067 bash[24488]: cluster 2024-02-10T08:26:26.626903+0000 mgr.y (mgr.14547) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:26 smithi134 bash[18722]: cluster 2024-02-10T08:26:26.626903+0000 mgr.y (mgr.14547) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:26:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:27] "GET /metrics HTTP/1.1" 200 172166 "" "Prometheus/2.33.4" 2024-02-10T08:26:29.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/2410109145 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880fdc90 msgr2=0x7f1e880fe070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:29.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/2410109145 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880fdc90 0x7f1e880fe070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f1e78004870 tx=0x7f1e78009cb0).stop 2024-02-10T08:26:29.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/2410109145 shutdown_connections 2024-02-10T08:26:29.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/2410109145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e880f8500 0x7f1e88104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.480 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/2410109145 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880fdc90 0x7f1e880fe070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/2410109145 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fe5f0 0x7f1e880f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/2410109145 >> 172.21.15.67:0/2410109145 conn(0x7f1e880f38c0 msgr2=0x7f1e880f5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:29.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/2410109145 shutdown_connections 2024-02-10T08:26:29.481 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/2410109145 wait complete. 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 Processor -- start 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- start start 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 0x7f1e8806b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fdc90 0x7f1e8806b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e880fe5f0 0x7f1e8806bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e88116490 con 0x7f1e880fdc90 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e88116600 con 0x7f1e880fe5f0 2024-02-10T08:26:29.482 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e8dbdb700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881167c0 con 0x7f1e880f8500 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e7effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fdc90 0x7f1e8806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 0x7f1e8806b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e7effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fdc90 0x7f1e8806b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:51138/0 (socket says 172.21.15.67:51138) 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 0x7f1e8806b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:35914/0 (socket says 172.21.15.67:35914) 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e7effd700 1 -- 172.21.15.67:0/1645198151 learned_addr learned my addr 172.21.15.67:0/1645198151 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e87fff700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e880fe5f0 0x7f1e8806bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 -- 172.21.15.67:0/1645198151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e880fe5f0 msgr2=0x7f1e8806bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e880fe5f0 0x7f1e8806bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 -- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fdc90 msgr2=0x7f1e8806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fdc90 0x7f1e8806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.483 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 -- 172.21.15.67:0/1645198151 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78005040 con 0x7f1e880f8500 2024-02-10T08:26:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.475+0000 7f1e877fe700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 0x7f1e8806b170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1e78004870 tx=0x7f1e78009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7800e030 con 0x7f1e880f8500 2024-02-10T08:26:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f1e78013db0 con 0x7f1e880f8500 2024-02-10T08:26:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78018070 con 0x7f1e880f8500 2024-02-10T08:26:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e88116a40 con 0x7f1e880f8500 2024-02-10T08:26:29.484 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e88116ed0 con 0x7f1e880f8500 2024-02-10T08:26:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88060740 con 0x7f1e880f8500 2024-02-10T08:26:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1e78009240 con 0x7f1e880f8500 2024-02-10T08:26:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e857fa700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e74061340 0x7f1e74063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:29.485 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f1e7807dfd0 con 0x7f1e880f8500 2024-02-10T08:26:29.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e7effd700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e74061340 0x7f1e74063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:29.486 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.479+0000 7f1e7effd700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e74061340 0x7f1e74063800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1e8806c910 tx=0x7f1e7000b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:29.492 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.487+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1e7804dd50 con 0x7f1e880f8500 2024-02-10T08:26:29.680 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.675+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e8806c650 con 0x7f1e880f8500 2024-02-10T08:26:29.681 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.675+0000 7f1e857fa700 1 -- 172.21.15.67:0/1645198151 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e780094f0 con 0x7f1e880f8500 2024-02-10T08:26:29.681 INFO:teuthology.orchestra.run.smithi067.stdout:107374182484 2024-02-10T08:26:29.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e74061340 msgr2=0x7f1e74063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:29.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e74061340 0x7f1e74063800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1e8806c910 tx=0x7f1e7000b040).stop 2024-02-10T08:26:29.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 msgr2=0x7f1e8806b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:29.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 0x7f1e8806b170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f1e78004870 tx=0x7f1e78009cb0).stop 2024-02-10T08:26:29.683 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 shutdown_connections 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e74061340 0x7f1e74063800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e880fe5f0 0x7f1e8806bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e880f8500 0x7f1e8806b170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 --2- 172.21.15.67:0/1645198151 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e880fdc90 0x7f1e8806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 >> 172.21.15.67:0/1645198151 conn(0x7f1e880f38c0 msgr2=0x7f1e880fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 shutdown_connections 2024-02-10T08:26:29.684 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:29.679+0000 7f1e8dbdb700 1 -- 172.21.15.67:0/1645198151 wait complete. 2024-02-10T08:26:29.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:29 smithi067 bash[17385]: cluster 2024-02-10T08:26:28.627536+0000 mgr.y (mgr.14547) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:29.947 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:29 smithi067 bash[17385]: audit 2024-02-10T08:26:29.680504+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? 172.21.15.67:0/1645198151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T08:26:29.947 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:29 smithi067 bash[24488]: cluster 2024-02-10T08:26:28.627536+0000 mgr.y (mgr.14547) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:29.947 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:29 smithi067 bash[24488]: audit 2024-02-10T08:26:29.680504+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? 172.21.15.67:0/1645198151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T08:26:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:29 smithi134 bash[18722]: cluster 2024-02-10T08:26:28.627536+0000 mgr.y (mgr.14547) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:29 smithi134 bash[18722]: audit 2024-02-10T08:26:29.680504+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? 172.21.15.67:0/1645198151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-10T08:26:30.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182471 got 107374182484 for osd.3 2024-02-10T08:26:30.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-10T08:26:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:30 smithi134 bash[18722]: cluster 2024-02-10T08:26:30.627994+0000 mgr.y (mgr.14547) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:30 smithi067 bash[17385]: cluster 2024-02-10T08:26:30.627994+0000 mgr.y (mgr.14547) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:30 smithi067 bash[24488]: cluster 2024-02-10T08:26:30.627994+0000 mgr.y (mgr.14547) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:32.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:26:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:26:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:26:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:26:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:33 smithi134 bash[18722]: audit 2024-02-10T08:26:31.745377+0000 mgr.y (mgr.14547) 154 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:33 smithi134 bash[18722]: cluster 2024-02-10T08:26:32.629417+0000 mgr.y (mgr.14547) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:33 smithi067 bash[17385]: audit 2024-02-10T08:26:31.745377+0000 mgr.y (mgr.14547) 154 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:33 smithi067 bash[17385]: cluster 2024-02-10T08:26:32.629417+0000 mgr.y (mgr.14547) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:33 smithi067 bash[24488]: audit 2024-02-10T08:26:31.745377+0000 mgr.y (mgr.14547) 154 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:33 smithi067 bash[24488]: cluster 2024-02-10T08:26:32.629417+0000 mgr.y (mgr.14547) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:34 smithi134 bash[18722]: cluster 2024-02-10T08:26:34.630118+0000 mgr.y (mgr.14547) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:34 smithi067 bash[17385]: cluster 2024-02-10T08:26:34.630118+0000 mgr.y (mgr.14547) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:34 smithi067 bash[24488]: cluster 2024-02-10T08:26:34.630118+0000 mgr.y (mgr.14547) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:35.225 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:36.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:36 smithi067 bash[24488]: cluster 2024-02-10T08:26:36.631553+0000 mgr.y (mgr.14547) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:36 smithi067 bash[17385]: cluster 2024-02-10T08:26:36.631553+0000 mgr.y (mgr.14547) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:36 smithi134 bash[18722]: cluster 2024-02-10T08:26:36.631553+0000 mgr.y (mgr.14547) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:26:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:37] "GET /metrics HTTP/1.1" 200 172164 "" "Prometheus/2.33.4" 2024-02-10T08:26:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 -- 172.21.15.67:0/4070549694 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fcbd0 msgr2=0x7f1e9c0fcfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 --2- 172.21.15.67:0/4070549694 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fcbd0 0x7f1e9c0fcfb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f1e98009c30 tx=0x7f1e98004c50).stop 2024-02-10T08:26:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 -- 172.21.15.67:0/4070549694 shutdown_connections 2024-02-10T08:26:38.161 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 --2- 172.21.15.67:0/4070549694 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 --2- 172.21.15.67:0/4070549694 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fcbd0 0x7f1e9c0fcfb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 --2- 172.21.15.67:0/4070549694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fd530 0x7f1e9c0f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 -- 172.21.15.67:0/4070549694 >> 172.21.15.67:0/4070549694 conn(0x7f1e9c070160 msgr2=0x7f1e9c0f44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 -- 172.21.15.67:0/4070549694 shutdown_connections 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 -- 172.21.15.67:0/4070549694 wait complete. 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 Processor -- start 2024-02-10T08:26:38.162 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.155+0000 7f1ea315e700 1 -- start start 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c110300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fcbd0 0x7f1e9c1107f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fd530 0x7f1e9c10a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c10a990 con 0x7f1e9c0f7960 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c10ab00 con 0x7f1e9c0fcbd0 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c10acc0 con 0x7f1e9c0fd530 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c110300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:38.163 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c110300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:40270/0 (socket says 172.21.15.67:40270) 2024-02-10T08:26:38.164 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 -- 172.21.15.67:0/1576479282 learned_addr learned my addr 172.21.15.67:0/1576479282 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea16fb700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fd530 0x7f1e9c10a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1e93fff700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fcbd0 0x7f1e9c1107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 -- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fd530 msgr2=0x7f1e9c10a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fd530 0x7f1e9c10a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 -- 172.21.15.67:0/1576479282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fcbd0 msgr2=0x7f1e9c1107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fcbd0 0x7f1e9c1107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 -- 172.21.15.67:0/1576479282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980097c0 con 0x7f1e9c0f7960 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1e93fff700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fcbd0 0x7f1e9c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea0efa700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c110300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f1e98004890 tx=0x7f1e98004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9800e040 con 0x7f1e9c0f7960 2024-02-10T08:26:38.165 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f1e98018040 con 0x7f1e9c0f7960 2024-02-10T08:26:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c10af40 con 0x7f1e9c0f7960 2024-02-10T08:26:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c117130 con 0x7f1e9c0f7960 2024-02-10T08:26:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98013c40 con 0x7f1e9c0f7960 2024-02-10T08:26:38.166 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.159+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88004f50 con 0x7f1e9c0f7960 2024-02-10T08:26:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.163+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f1e98013420 con 0x7f1e9c0f7960 2024-02-10T08:26:38.168 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.163+0000 7f1e91ffb700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e84061340 0x7f1e84063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.163+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f1e9807e000 con 0x7f1e9c0f7960 2024-02-10T08:26:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.163+0000 7f1e93fff700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e84061340 0x7f1e84063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.163+0000 7f1e93fff700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e84061340 0x7f1e84063800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f1e9c10bc40 tx=0x7f1e8c00b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:38.174 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.167+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f1e9804def0 con 0x7f1e9c0f7960 2024-02-10T08:26:38.366 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.359+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1e880054d0 con 0x7f1e9c0f7960 2024-02-10T08:26:38.367 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1e91ffb700 1 -- 172.21.15.67:0/1576479282 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e98021020 con 0x7f1e9c0f7960 2024-02-10T08:26:38.367 INFO:teuthology.orchestra.run.smithi067.stdout:128849018957 2024-02-10T08:26:38.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e84061340 msgr2=0x7f1e84063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:38.370 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e84061340 0x7f1e84063800 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f1e9c10bc40 tx=0x7f1e8c00b040).stop 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 msgr2=0x7f1e9c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c110300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f1e98004890 tx=0x7f1e98004c50).stop 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 shutdown_connections 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f1e9c0f7960 0x7f1e9c110300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1e84061340 0x7f1e84063800 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f1e9c0fd530 0x7f1e9c10a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 --2- 172.21.15.67:0/1576479282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e9c0fcbd0 0x7f1e9c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 >> 172.21.15.67:0/1576479282 conn(0x7f1e9c070160 msgr2=0x7f1e9c0f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 shutdown_connections 2024-02-10T08:26:38.371 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:38.363+0000 7f1ea315e700 1 -- 172.21.15.67:0/1576479282 wait complete. 2024-02-10T08:26:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:38 smithi134 bash[18722]: audit 2024-02-10T08:26:37.679826+0000 mon.c (mon.2) 78 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:26:38.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:38 smithi134 bash[18722]: audit 2024-02-10T08:26:37.952881+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:26:38.594 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:38 smithi067 bash[17385]: audit 2024-02-10T08:26:37.679826+0000 mon.c (mon.2) 78 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:26:38.595 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:38 smithi067 bash[17385]: audit 2024-02-10T08:26:37.952881+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:26:38.595 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:38 smithi067 bash[24488]: audit 2024-02-10T08:26:37.679826+0000 mon.c (mon.2) 78 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:26:38.595 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:38 smithi067 bash[24488]: audit 2024-02-10T08:26:37.952881+0000 mon.a (mon.0) 682 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:26:39.112 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018943 got 128849018957 for osd.4 2024-02-10T08:26:39.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-10T08:26:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[17385]: audit 2024-02-10T08:26:38.249838+0000 mon.c (mon.2) 79 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:26:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[17385]: audit 2024-02-10T08:26:38.251486+0000 mon.c (mon.2) 80 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:26:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[17385]: audit 2024-02-10T08:26:38.259087+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:26:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[17385]: audit 2024-02-10T08:26:38.366642+0000 mon.a (mon.0) 684 : audit [DBG] from='client.? 172.21.15.67:0/1576479282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T08:26:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[17385]: cluster 2024-02-10T08:26:38.632074+0000 mgr.y (mgr.14547) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[24488]: audit 2024-02-10T08:26:38.249838+0000 mon.c (mon.2) 79 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:26:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[24488]: audit 2024-02-10T08:26:38.251486+0000 mon.c (mon.2) 80 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:26:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[24488]: audit 2024-02-10T08:26:38.259087+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:26:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[24488]: audit 2024-02-10T08:26:38.366642+0000 mon.a (mon.0) 684 : audit [DBG] from='client.? 172.21.15.67:0/1576479282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T08:26:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:39 smithi067 bash[24488]: cluster 2024-02-10T08:26:38.632074+0000 mgr.y (mgr.14547) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:39 smithi134 bash[18722]: audit 2024-02-10T08:26:38.249838+0000 mon.c (mon.2) 79 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:26:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:39 smithi134 bash[18722]: audit 2024-02-10T08:26:38.251486+0000 mon.c (mon.2) 80 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:26:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:39 smithi134 bash[18722]: audit 2024-02-10T08:26:38.259087+0000 mon.a (mon.0) 683 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:26:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:39 smithi134 bash[18722]: audit 2024-02-10T08:26:38.366642+0000 mon.a (mon.0) 684 : audit [DBG] from='client.? 172.21.15.67:0/1576479282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-10T08:26:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:39 smithi134 bash[18722]: cluster 2024-02-10T08:26:38.632074+0000 mgr.y (mgr.14547) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:40 smithi134 bash[18722]: cluster 2024-02-10T08:26:40.632723+0000 mgr.y (mgr.14547) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:40 smithi067 bash[17385]: cluster 2024-02-10T08:26:40.632723+0000 mgr.y (mgr.14547) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:40 smithi067 bash[24488]: cluster 2024-02-10T08:26:40.632723+0000 mgr.y (mgr.14547) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:42.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:26:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:26:43.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:26:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:26:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:43 smithi134 bash[18722]: audit 2024-02-10T08:26:41.751577+0000 mgr.y (mgr.14547) 160 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:43 smithi134 bash[18722]: cluster 2024-02-10T08:26:42.634200+0000 mgr.y (mgr.14547) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:43 smithi067 bash[17385]: audit 2024-02-10T08:26:41.751577+0000 mgr.y (mgr.14547) 160 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:43 smithi067 bash[17385]: cluster 2024-02-10T08:26:42.634200+0000 mgr.y (mgr.14547) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:43 smithi067 bash[24488]: audit 2024-02-10T08:26:41.751577+0000 mgr.y (mgr.14547) 160 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:43 smithi067 bash[24488]: cluster 2024-02-10T08:26:42.634200+0000 mgr.y (mgr.14547) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:43.899 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:45.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:44 smithi134 bash[18722]: cluster 2024-02-10T08:26:44.634731+0000 mgr.y (mgr.14547) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:45.084 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:44 smithi067 bash[17385]: cluster 2024-02-10T08:26:44.634731+0000 mgr.y (mgr.14547) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:45.085 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:44 smithi067 bash[24488]: cluster 2024-02-10T08:26:44.634731+0000 mgr.y (mgr.14547) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/2103278183 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c00f8c90 msgr2=0x7fb1c00f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- 172.21.15.67:0/2103278183 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c00f8c90 0x7fb1c00f9070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fb1b0009be0 tx=0x7fb1b0004d10).stop 2024-02-10T08:26:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/2103278183 shutdown_connections 2024-02-10T08:26:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- 172.21.15.67:0/2103278183 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c01021f0 0x7fb1c01045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- 172.21.15.67:0/2103278183 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c00f8c90 0x7fb1c00f9070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:46.824 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- 172.21.15.67:0/2103278183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00f95f0 0x7fb1c0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:46.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/2103278183 >> 172.21.15.67:0/2103278183 conn(0x7fb1c00f27a0 msgr2=0x7fb1c00f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:46.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/2103278183 shutdown_connections 2024-02-10T08:26:46.825 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/2103278183 wait complete. 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 Processor -- start 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- start start 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 0x7fb1c010e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00f95f0 0x7fb1c010e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c01021f0 0x7fb1c0108330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0108820 con 0x7fb1c00f8c90 2024-02-10T08:26:46.826 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0108990 con 0x7fb1c00f95f0 2024-02-10T08:26:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1c4b2a700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0108b50 con 0x7fb1c01021f0 2024-02-10T08:26:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 0x7fb1c010e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 0x7fb1c010e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:57072/0 (socket says 172.21.15.67:57072) 2024-02-10T08:26:46.827 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 -- 172.21.15.67:0/3335094821 learned_addr learned my addr 172.21.15.67:0/3335094821 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1bed9d700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c01021f0 0x7fb1c0108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 -- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c01021f0 msgr2=0x7fb1c0108330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c01021f0 0x7fb1c0108330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 -- 172.21.15.67:0/3335094821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00f95f0 msgr2=0x7fb1c010e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00f95f0 0x7fb1c010e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 -- 172.21.15.67:0/3335094821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00097c0 con 0x7fb1c00f8c90 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.818+0000 7fb1be59c700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 0x7fb1c010e140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fb1b0003380 tx=0x7fb1b0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:46.828 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000e040 con 0x7fb1c00f8c90 2024-02-10T08:26:46.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fb1b0018040 con 0x7fb1c00f8c90 2024-02-10T08:26:46.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/3335094821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c0108dd0 con 0x7fb1c00f8c90 2024-02-10T08:26:46.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0013c70 con 0x7fb1c00f8c90 2024-02-10T08:26:46.829 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1c4b2a700 1 -- 172.21.15.67:0/3335094821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c0114f70 con 0x7fb1c00f8c90 2024-02-10T08:26:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a4004f20 con 0x7fb1c00f8c90 2024-02-10T08:26:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fb1b0013420 con 0x7fb1c00f8c90 2024-02-10T08:26:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1af7fe700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb1a8061340 0x7fb1a8063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7fb1b007e890 con 0x7fb1c00f8c90 2024-02-10T08:26:46.830 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.822+0000 7fb1bdd9b700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb1a8061340 0x7fb1a8063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:46.831 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.826+0000 7fb1bdd9b700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb1a8061340 0x7fb1a8063800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb1c0109a80 tx=0x7fb1b400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:46.837 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:46.830+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fb1b004e860 con 0x7fb1c00f8c90 2024-02-10T08:26:47.026 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.018+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb1a40051d0 con 0x7fb1c00f8c90 2024-02-10T08:26:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:46 smithi134 bash[18722]: cluster 2024-02-10T08:26:46.635961+0000 mgr.y (mgr.14547) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:47.027 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.018+0000 7fb1af7fe700 1 -- 172.21.15.67:0/3335094821 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1b0011320 con 0x7fb1c00f8c90 2024-02-10T08:26:47.027 INFO:teuthology.orchestra.run.smithi067.stdout:154618822729 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb1a8061340 msgr2=0x7fb1a8063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb1a8061340 0x7fb1a8063800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb1c0109a80 tx=0x7fb1b400b040).stop 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 msgr2=0x7fb1c010e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 0x7fb1c010e140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7fb1b0003380 tx=0x7fb1b0004d10).stop 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 shutdown_connections 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fb1c00f8c90 0x7fb1c010e140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fb1a8061340 0x7fb1a8063800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:47.030 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fb1c01021f0 0x7fb1c0108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 --2- 172.21.15.67:0/3335094821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1c00f95f0 0x7fb1c010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2024-02-10T08:26:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 >> 172.21.15.67:0/3335094821 conn(0x7fb1c00f27a0 msgr2=0x7fb1c00f34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 shutdown_connections 2024-02-10T08:26:47.031 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:47.022+0000 7fb1ad7fa700 1 -- 172.21.15.67:0/3335094821 wait complete. 2024-02-10T08:26:47.061 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:46 smithi067 bash[24488]: cluster 2024-02-10T08:26:46.635961+0000 mgr.y (mgr.14547) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:47.061 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:46 smithi067 bash[17385]: cluster 2024-02-10T08:26:46.635961+0000 mgr.y (mgr.14547) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:47.795 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822715 got 154618822729 for osd.5 2024-02-10T08:26:47.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-10T08:26:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:47 smithi134 bash[18722]: audit 2024-02-10T08:26:47.026423+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? 172.21.15.67:0/3335094821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T08:26:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:47 smithi067 bash[17385]: audit 2024-02-10T08:26:47.026423+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? 172.21.15.67:0/3335094821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T08:26:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:47 smithi067 bash[24488]: audit 2024-02-10T08:26:47.026423+0000 mon.a (mon.0) 685 : audit [DBG] from='client.? 172.21.15.67:0/3335094821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-10T08:26:48.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:26:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:47] "GET /metrics HTTP/1.1" 200 172183 "" "Prometheus/2.33.4" 2024-02-10T08:26:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:48 smithi134 bash[18722]: cluster 2024-02-10T08:26:48.636568+0000 mgr.y (mgr.14547) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:48 smithi067 bash[17385]: cluster 2024-02-10T08:26:48.636568+0000 mgr.y (mgr.14547) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:49.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:48 smithi067 bash[24488]: cluster 2024-02-10T08:26:48.636568+0000 mgr.y (mgr.14547) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:50 smithi134 bash[18722]: cluster 2024-02-10T08:26:50.637287+0000 mgr.y (mgr.14547) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:51.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:50 smithi067 bash[17385]: cluster 2024-02-10T08:26:50.637287+0000 mgr.y (mgr.14547) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:51.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:50 smithi067 bash[24488]: cluster 2024-02-10T08:26:50.637287+0000 mgr.y (mgr.14547) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:52.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:26:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:26:52.585 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:26:53.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:26:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:26:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:53 smithi134 bash[18722]: audit 2024-02-10T08:26:51.763031+0000 mgr.y (mgr.14547) 166 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:53 smithi134 bash[18722]: cluster 2024-02-10T08:26:52.638830+0000 mgr.y (mgr.14547) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:53.545 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:53 smithi067 bash[24488]: audit 2024-02-10T08:26:51.763031+0000 mgr.y (mgr.14547) 166 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:53.545 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:53 smithi067 bash[24488]: cluster 2024-02-10T08:26:52.638830+0000 mgr.y (mgr.14547) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:53.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:53 smithi067 bash[17385]: audit 2024-02-10T08:26:51.763031+0000 mgr.y (mgr.14547) 166 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:26:53.545 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:53 smithi067 bash[17385]: cluster 2024-02-10T08:26:52.638830+0000 mgr.y (mgr.14547) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:55.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:54 smithi134 bash[18722]: cluster 2024-02-10T08:26:54.639319+0000 mgr.y (mgr.14547) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:55.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:54 smithi067 bash[17385]: cluster 2024-02-10T08:26:54.639319+0000 mgr.y (mgr.14547) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:55.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:54 smithi067 bash[24488]: cluster 2024-02-10T08:26:54.639319+0000 mgr.y (mgr.14547) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:55.503 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/4064592388 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef40f8c70 msgr2=0x7f6ef40f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/4064592388 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef40f8c70 0x7f6ef40f9050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6ef0004870 tx=0x7f6ef0009cb0).stop 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/4064592388 shutdown_connections 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/4064592388 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef41021d0 0x7f6ef41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/4064592388 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef40f8c70 0x7f6ef40f9050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/4064592388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40f95d0 0x7f6ef4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/4064592388 >> 172.21.15.67:0/4064592388 conn(0x7f6ef40f2760 msgr2=0x7f6ef40f4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/4064592388 shutdown_connections 2024-02-10T08:26:55.504 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/4064592388 wait complete. 2024-02-10T08:26:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 Processor -- start 2024-02-10T08:26:55.505 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- start start 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 0x7f6ef410e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40f95d0 0x7f6ef410e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef41021d0 0x7f6ef41082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6ef9163700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 0x7f6ef410e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6ef9163700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 0x7f6ef410e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47732/0 (socket says 172.21.15.67:47732) 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6ef9163700 1 -- 172.21.15.67:0/2937940133 learned_addr learned my addr 172.21.15.67:0/2937940133 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef41087b0 con 0x7f6ef40f8c70 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4108920 con 0x7f6ef40f95d0 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef4108ae0 con 0x7f6ef41021d0 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6ef9964700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef41021d0 0x7f6ef41082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:55.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.498+0000 7f6ef8962700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40f95d0 0x7f6ef410e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:55.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9964700 1 -- 172.21.15.67:0/2937940133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40f95d0 msgr2=0x7f6ef410e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:55.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9964700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40f95d0 0x7f6ef410e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9964700 1 -- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 msgr2=0x7f6ef410e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9964700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 0x7f6ef410e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9964700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef0005040 con 0x7f6ef41021d0 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9163700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 0x7f6ef410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9964700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef41021d0 0x7f6ef41082c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6eec00eb40 tx=0x7f6eec00b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec009870 con 0x7f6ef41021d0 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f6eec00f040 con 0x7f6ef41021d0 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec007c00 con 0x7f6ef41021d0 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef4108d60 con 0x7f6ef41021d0 2024-02-10T08:26:55.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef4114f00 con 0x7f6ef41021d0 2024-02-10T08:26:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef405faf0 con 0x7f6ef41021d0 2024-02-10T08:26:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6eec0092e0 con 0x7f6ef41021d0 2024-02-10T08:26:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6eea7fc700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6ee0061340 0x7f6ee0063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:26:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f6eec07cc50 con 0x7f6ef41021d0 2024-02-10T08:26:55.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.502+0000 7f6ef9163700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6ee0061340 0x7f6ee0063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:26:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.506+0000 7f6ef9163700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6ee0061340 0x7f6ee0063800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f6ef0005010 tx=0x7f6ef0009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:26:55.517 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.510+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6eec049f60 con 0x7f6ef41021d0 2024-02-10T08:26:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.690+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6ef4106fc0 con 0x7f6ef41021d0 2024-02-10T08:26:55.697 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.690+0000 7f6eea7fc700 1 -- 172.21.15.67:0/2937940133 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6eec07ae60 con 0x7f6ef41021d0 2024-02-10T08:26:55.698 INFO:teuthology.orchestra.run.smithi067.stdout:180388626497 2024-02-10T08:26:55.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6ee0061340 msgr2=0x7f6ee0063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:55.700 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6ee0061340 0x7f6ee0063800 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f6ef0005010 tx=0x7f6ef0009cb0).stop 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef41021d0 msgr2=0x7f6ef41082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef41021d0 0x7f6ef41082c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f6eec00eb40 tx=0x7f6eec00b040).stop 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 shutdown_connections 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6ef40f8c70 0x7f6ef410e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6ee0061340 0x7f6ee0063800 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6ef41021d0 0x7f6ef41082c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 --2- 172.21.15.67:0/2937940133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ef40f95d0 0x7f6ef410e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 >> 172.21.15.67:0/2937940133 conn(0x7f6ef40f2760 msgr2=0x7f6ef40f3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 shutdown_connections 2024-02-10T08:26:55.701 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:26:55.694+0000 7f6efb3c7700 1 -- 172.21.15.67:0/2937940133 wait complete. 2024-02-10T08:26:56.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:55 smithi134 bash[18722]: audit 2024-02-10T08:26:55.697589+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? 172.21.15.67:0/2937940133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T08:26:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:55 smithi067 bash[17385]: audit 2024-02-10T08:26:55.697589+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? 172.21.15.67:0/2937940133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T08:26:56.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:55 smithi067 bash[24488]: audit 2024-02-10T08:26:55.697589+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? 172.21.15.67:0/2937940133' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-10T08:26:56.507 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626484 got 180388626497 for osd.6 2024-02-10T08:26:56.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-10T08:26:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:56 smithi067 bash[17385]: cluster 2024-02-10T08:26:56.640682+0000 mgr.y (mgr.14547) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:56.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:56 smithi067 bash[24488]: cluster 2024-02-10T08:26:56.640682+0000 mgr.y (mgr.14547) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:56 smithi134 bash[18722]: cluster 2024-02-10T08:26:56.640682+0000 mgr.y (mgr.14547) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:26:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:26:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:26:57] "GET /metrics HTTP/1.1" 200 172183 "" "Prometheus/2.33.4" 2024-02-10T08:26:59.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:26:58 smithi134 bash[18722]: cluster 2024-02-10T08:26:58.641346+0000 mgr.y (mgr.14547) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:59.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:26:58 smithi067 bash[17385]: cluster 2024-02-10T08:26:58.641346+0000 mgr.y (mgr.14547) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:26:59.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:26:58 smithi067 bash[24488]: cluster 2024-02-10T08:26:58.641346+0000 mgr.y (mgr.14547) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:01.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:00 smithi134 bash[18722]: cluster 2024-02-10T08:27:00.642006+0000 mgr.y (mgr.14547) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:00 smithi067 bash[17385]: cluster 2024-02-10T08:27:00.642006+0000 mgr.y (mgr.14547) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:01.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:00 smithi067 bash[24488]: cluster 2024-02-10T08:27:00.642006+0000 mgr.y (mgr.14547) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:01.285 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:27:02.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:27:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:27:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:03 smithi134 bash[18722]: audit 2024-02-10T08:27:01.774459+0000 mgr.y (mgr.14547) 172 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:03 smithi134 bash[18722]: cluster 2024-02-10T08:27:02.642860+0000 mgr.y (mgr.14547) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:03.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:27:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:27:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:03 smithi067 bash[17385]: audit 2024-02-10T08:27:01.774459+0000 mgr.y (mgr.14547) 172 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:03 smithi067 bash[17385]: cluster 2024-02-10T08:27:02.642860+0000 mgr.y (mgr.14547) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:03 smithi067 bash[24488]: audit 2024-02-10T08:27:01.774459+0000 mgr.y (mgr.14547) 172 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:03 smithi067 bash[24488]: cluster 2024-02-10T08:27:02.642860+0000 mgr.y (mgr.14547) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:04.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 -- 172.21.15.67:0/110381529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6df0 msgr2=0x7fa134106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:04.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 --2- 172.21.15.67:0/110381529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6df0 0x7fa134106c10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7fa12c009be0 tx=0x7fa12c004d10).stop 2024-02-10T08:27:04.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 -- 172.21.15.67:0/110381529 shutdown_connections 2024-02-10T08:27:04.049 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 --2- 172.21.15.67:0/110381529 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6df0 0x7fa134106c10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 --2- 172.21.15.67:0/110381529 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f8ab0 0x7fa1340f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 --2- 172.21.15.67:0/110381529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1340f6480 0x7fa1340f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 -- 172.21.15.67:0/110381529 >> 172.21.15.67:0/110381529 conn(0x7fa13406c400 msgr2=0x7fa13406c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 -- 172.21.15.67:0/110381529 shutdown_connections 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 -- 172.21.15.67:0/110381529 wait complete. 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.046+0000 7fa13994d700 1 Processor -- start 2024-02-10T08:27:04.050 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 -- start start 2024-02-10T08:27:04.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6480 0x7fa134110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:04.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 0x7fa1341105f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:04.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1340f8ab0 0x7fa13410a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:04.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13410a790 con 0x7fa1340f6480 2024-02-10T08:27:04.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13410a900 con 0x7fa1340f8ab0 2024-02-10T08:27:04.051 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13410aac0 con 0x7fa1340f6df0 2024-02-10T08:27:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 0x7fa1341105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa132ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6480 0x7fa134110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 0x7fa1341105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:52172/0 (socket says 172.21.15.67:52172) 2024-02-10T08:27:04.052 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 -- 172.21.15.67:0/3301306393 learned_addr learned my addr 172.21.15.67:0/3301306393 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1337fe700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1340f8ab0 0x7fa13410a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 -- 172.21.15.67:0/3301306393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1340f8ab0 msgr2=0x7fa13410a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1340f8ab0 0x7fa13410a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 -- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6480 msgr2=0x7fa134110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6480 0x7fa134110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 -- 172.21.15.67:0/3301306393 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0097c0 con 0x7fa1340f6df0 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa132ffd700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6480 0x7fa134110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa1327fc700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 0x7fa1341105f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa12800e8f0 tx=0x7fa12800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:04.053 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128004660 con 0x7fa1340f6df0 2024-02-10T08:27:04.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fa12800f040 con 0x7fa1340f6df0 2024-02-10T08:27:04.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280146f0 con 0x7fa1340f6df0 2024-02-10T08:27:04.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13410ada0 con 0x7fa1340f6df0 2024-02-10T08:27:04.054 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.050+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13410b120 con 0x7fa1340f6df0 2024-02-10T08:27:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.054+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fa128007b40 con 0x7fa1340f6df0 2024-02-10T08:27:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.054+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13405fa70 con 0x7fa1340f6df0 2024-02-10T08:27:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.054+0000 7fa11bfff700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa11c061340 0x7fa11c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:04.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.054+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7fa12807c990 con 0x7fa1340f6df0 2024-02-10T08:27:04.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.054+0000 7fa132ffd700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa11c061340 0x7fa11c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:04.056 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.054+0000 7fa132ffd700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa11c061340 0x7fa11c063800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa124007b50 tx=0x7fa12400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:04.061 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.058+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fa12804c7a0 con 0x7fa1340f6df0 2024-02-10T08:27:04.247 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.246+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa1340475e0 con 0x7fa1340f6df0 2024-02-10T08:27:04.248 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.246+0000 7fa11bfff700 1 -- 172.21.15.67:0/3301306393 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa128019070 con 0x7fa1340f6df0 2024-02-10T08:27:04.248 INFO:teuthology.orchestra.run.smithi067.stdout:206158430267 2024-02-10T08:27:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa11c061340 msgr2=0x7fa11c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa11c061340 0x7fa11c063800 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fa124007b50 tx=0x7fa12400b040).stop 2024-02-10T08:27:04.252 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 msgr2=0x7fa1341105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 0x7fa1341105f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fa12800e8f0 tx=0x7fa12800b040).stop 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 shutdown_connections 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fa1340f6480 0x7fa134110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fa11c061340 0x7fa11c063800 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1340f8ab0 0x7fa13410a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 --2- 172.21.15.67:0/3301306393 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fa1340f6df0 0x7fa1341105f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 >> 172.21.15.67:0/3301306393 conn(0x7fa13406c400 msgr2=0x7fa1340f4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 shutdown_connections 2024-02-10T08:27:04.253 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:04.250+0000 7fa13994d700 1 -- 172.21.15.67:0/3301306393 wait complete. 2024-02-10T08:27:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:04 smithi067 bash[17385]: audit 2024-02-10T08:27:04.248128+0000 mon.c (mon.2) 82 : audit [DBG] from='client.? 172.21.15.67:0/3301306393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T08:27:04.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:04 smithi067 bash[24488]: audit 2024-02-10T08:27:04.248128+0000 mon.c (mon.2) 82 : audit [DBG] from='client.? 172.21.15.67:0/3301306393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T08:27:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:04 smithi134 bash[18722]: audit 2024-02-10T08:27:04.248128+0000 mon.c (mon.2) 82 : audit [DBG] from='client.? 172.21.15.67:0/3301306393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-10T08:27:04.990 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430253 got 206158430267 for osd.7 2024-02-10T08:27:04.990 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-10T08:27:04.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T08:27:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:05 smithi067 bash[24488]: cluster 2024-02-10T08:27:04.643481+0000 mgr.y (mgr.14547) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:05 smithi067 bash[17385]: cluster 2024-02-10T08:27:04.643481+0000 mgr.y (mgr.14547) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:05.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:05 smithi134 bash[18722]: cluster 2024-02-10T08:27:04.643481+0000 mgr.y (mgr.14547) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:06 smithi134 bash[18722]: cluster 2024-02-10T08:27:06.644992+0000 mgr.y (mgr.14547) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:07.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:06 smithi067 bash[17385]: cluster 2024-02-10T08:27:06.644992+0000 mgr.y (mgr.14547) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:07.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:06 smithi067 bash[24488]: cluster 2024-02-10T08:27:06.644992+0000 mgr.y (mgr.14547) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:27:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:07] "GET /metrics HTTP/1.1" 200 172192 "" "Prometheus/2.33.4" 2024-02-10T08:27:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:08 smithi134 bash[18722]: cluster 2024-02-10T08:27:08.645811+0000 mgr.y (mgr.14547) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:09.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:08 smithi067 bash[17385]: cluster 2024-02-10T08:27:08.645811+0000 mgr.y (mgr.14547) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:08 smithi067 bash[24488]: cluster 2024-02-10T08:27:08.645811+0000 mgr.y (mgr.14547) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:09.782 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:27:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:10 smithi067 bash[24488]: cluster 2024-02-10T08:27:10.646481+0000 mgr.y (mgr.14547) 177 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:10 smithi067 bash[17385]: cluster 2024-02-10T08:27:10.646481+0000 mgr.y (mgr.14547) 177 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:11.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:10 smithi134 bash[18722]: cluster 2024-02-10T08:27:10.646481+0000 mgr.y (mgr.14547) 177 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:12.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:27:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 -- 172.21.15.67:0/1558687141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0f7400 msgr2=0x7fc88c103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 --2- 172.21.15.67:0/1558687141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0f7400 0x7fc88c103a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fc884009be0 tx=0x7fc884004d10).stop 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 -- 172.21.15.67:0/1558687141 shutdown_connections 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 --2- 172.21.15.67:0/1558687141 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0f7400 0x7fc88c103a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 --2- 172.21.15.67:0/1558687141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c0fcb90 0x7fc88c0fcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 --2- 172.21.15.67:0/1558687141 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0fd4f0 0x7fc88c0f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 -- 172.21.15.67:0/1558687141 >> 172.21.15.67:0/1558687141 conn(0x7fc88c0f27a0 msgr2=0x7fc88c0f4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 -- 172.21.15.67:0/1558687141 shutdown_connections 2024-02-10T08:27:12.506 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.502+0000 7fc892534700 1 -- 172.21.15.67:0/1558687141 wait complete. 2024-02-10T08:27:12.507 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 Processor -- start 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 -- start start 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 0x7fc88c10dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c0fcb90 0x7fc88c10e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 0x7fc88c107f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c108400 con 0x7fc88c0fd4f0 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c108570 con 0x7fc88c0fcb90 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c108730 con 0x7fc88c0f7400 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 0x7fc88c10dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc890ad1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 0x7fc88c107f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc890ad1700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 0x7fc88c107f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:43116/0 (socket says 172.21.15.67:43116) 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 0x7fc88c10dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:52182/0 (socket says 172.21.15.67:52182) 2024-02-10T08:27:12.508 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc890ad1700 1 -- 172.21.15.67:0/2383157577 learned_addr learned my addr 172.21.15.67:0/2383157577 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:27:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc8837fe700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c0fcb90 0x7fc88c10e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 -- 172.21.15.67:0/2383157577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c0fcb90 msgr2=0x7fc88c10e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:12.509 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c0fcb90 0x7fc88c10e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 -- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 msgr2=0x7fc88c107f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 0x7fc88c107f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 -- 172.21.15.67:0/2383157577 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840097c0 con 0x7fc88c0f7400 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc890ad1700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 0x7fc88c107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc88bfff700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 0x7fc88c10dd70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc87c002800 tx=0x7fc87c009ec0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c007b40 con 0x7fc88c0f7400 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c108a10 con 0x7fc88c0f7400 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fc87c00f040 con 0x7fc88c0f7400 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c014650 con 0x7fc88c0f7400 2024-02-10T08:27:12.510 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.506+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c114ba0 con 0x7fc88c0f7400 2024-02-10T08:27:12.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.510+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fc87c009110 con 0x7fc88c0f7400 2024-02-10T08:27:12.511 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.510+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c060740 con 0x7fc88c0f7400 2024-02-10T08:27:12.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.510+0000 7fc889ffb700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc878061210 0x7fc8780636d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:12.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.510+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7fc87c07d100 con 0x7fc88c0f7400 2024-02-10T08:27:12.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.510+0000 7fc8837fe700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc878061210 0x7fc8780636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:12.512 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.510+0000 7fc8837fe700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc878061210 0x7fc8780636d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fc88c1096b0 tx=0x7fc87400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:12.518 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.518+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fc87c04ce80 con 0x7fc88c0f7400 2024-02-10T08:27:12.707 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.706+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 --> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc88c109500 con 0x7fc878061210 2024-02-10T08:27:12.728 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.726+0000 7fc889ffb700 1 -- 172.21.15.67:0/2383157577 <== mgr.14547 v2:172.21.15.67:6800/4090043441 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355366 (secure 0 0 0) 0x7fc88c109500 con 0x7fc878061210 2024-02-10T08:27:12.734 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc878061210 msgr2=0x7fc8780636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc878061210 0x7fc8780636d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fc88c1096b0 tx=0x7fc87400b040).stop 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 msgr2=0x7fc88c10dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 0x7fc88c10dd70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fc87c002800 tx=0x7fc87c009ec0).stop 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 shutdown_connections 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fc88c0fd4f0 0x7fc88c107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fc878061210 0x7fc8780636d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fc88c0f7400 0x7fc88c10dd70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.737 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 --2- 172.21.15.67:0/2383157577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc88c0fcb90 0x7fc88c10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:12.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 >> 172.21.15.67:0/2383157577 conn(0x7fc88c0f27a0 msgr2=0x7fc88c0f4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:12.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 shutdown_connections 2024-02-10T08:27:12.738 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:12.730+0000 7fc892534700 1 -- 172.21.15.67:0/2383157577 wait complete. 2024-02-10T08:27:12.738 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-10T08:27:13.429 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-02-10T08:27:12.646761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":987,"num_read_kb":739,"num_write":428,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":466,"ondisk_log_size":466,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2382236,"kb_used_data":5468,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747578980,"statfs":{"total":767960285184,"available":765520875520,"internally_reserved":0,"allocated":5599232,"data_stored":2310872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005138"},"pg_stats":[{"pgid":"6.1a","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796133+0000","last_change":"2024-02-10T08:22:41.912064+0000","last_active":"2024-02-10T08:24:00.796133+0000","last_peered":"2024-02-10T08:24:00.796133+0000","last_clean":"2024-02-10T08:24:00.796133+0000","last_became_active":"2024-02-10T08:22:41.911671+0000","last_became_peered":"2024-02-10T08:22:41.911671+0000","last_unstale":"2024-02-10T08:24:00.796133+0000","last_undegraded":"2024-02-10T08:24:00.796133+0000","last_fullsized":"2024-02-10T08:24:00.796133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795122+0000","last_change":"2024-02-10T08:22:39.859638+0000","last_active":"2024-02-10T08:24:00.795122+0000","last_peered":"2024-02-10T08:24:00.795122+0000","last_clean":"2024-02-10T08:24:00.795122+0000","last_became_active":"2024-02-10T08:22:39.859323+0000","last_became_peered":"2024-02-10T08:22:39.859323+0000","last_unstale":"2024-02-10T08:24:00.795122+0000","last_undegraded":"2024-02-10T08:24:00.795122+0000","last_fullsized":"2024-02-10T08:24:00.795122+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790931+0000","last_change":"2024-02-10T08:22:35.811855+0000","last_active":"2024-02-10T08:24:00.790931+0000","last_peered":"2024-02-10T08:24:00.790931+0000","last_clean":"2024-02-10T08:24:00.790931+0000","last_became_active":"2024-02-10T08:22:35.811434+0000","last_became_peered":"2024-02-10T08:22:35.811434+0000","last_unstale":"2024-02-10T08:24:00.790931+0000","last_undegraded":"2024-02-10T08:24:00.790931+0000","last_fullsized":"2024-02-10T08:24:00.790931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796226+0000","last_change":"2024-02-10T08:22:37.824213+0000","last_active":"2024-02-10T08:24:00.796226+0000","last_peered":"2024-02-10T08:24:00.796226+0000","last_clean":"2024-02-10T08:24:00.796226+0000","last_became_active":"2024-02-10T08:22:37.823869+0000","last_became_peered":"2024-02-10T08:22:37.823869+0000","last_unstale":"2024-02-10T08:24:00.796226+0000","last_undegraded":"2024-02-10T08:24:00.796226+0000","last_fullsized":"2024-02-10T08:24:00.796226+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788287+0000","last_change":"2024-02-10T08:22:41.903827+0000","last_active":"2024-02-10T08:24:00.788287+0000","last_peered":"2024-02-10T08:24:00.788287+0000","last_clean":"2024-02-10T08:24:00.788287+0000","last_became_active":"2024-02-10T08:22:41.903451+0000","last_became_peered":"2024-02-10T08:22:41.903451+0000","last_unstale":"2024-02-10T08:24:00.788287+0000","last_undegraded":"2024-02-10T08:24:00.788287+0000","last_fullsized":"2024-02-10T08:24:00.788287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795972+0000","last_change":"2024-02-10T08:22:39.848043+0000","last_active":"2024-02-10T08:24:00.795972+0000","last_peered":"2024-02-10T08:24:00.795972+0000","last_clean":"2024-02-10T08:24:00.795972+0000","last_became_active":"2024-02-10T08:22:39.847674+0000","last_became_peered":"2024-02-10T08:22:39.847674+0000","last_unstale":"2024-02-10T08:24:00.795972+0000","last_undegraded":"2024-02-10T08:24:00.795972+0000","last_fullsized":"2024-02-10T08:24:00.795972+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789074+0000","last_change":"2024-02-10T08:22:35.813711+0000","last_active":"2024-02-10T08:24:00.789074+0000","last_peered":"2024-02-10T08:24:00.789074+0000","last_clean":"2024-02-10T08:24:00.789074+0000","last_became_active":"2024-02-10T08:22:35.813335+0000","last_became_peered":"2024-02-10T08:22:35.813335+0000","last_unstale":"2024-02-10T08:24:00.789074+0000","last_undegraded":"2024-02-10T08:24:00.789074+0000","last_fullsized":"2024-02-10T08:24:00.789074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788830+0000","last_change":"2024-02-10T08:22:37.830239+0000","last_active":"2024-02-10T08:24:00.788830+0000","last_peered":"2024-02-10T08:24:00.788830+0000","last_clean":"2024-02-10T08:24:00.788830+0000","last_became_active":"2024-02-10T08:22:37.829986+0000","last_became_peered":"2024-02-10T08:22:37.829986+0000","last_unstale":"2024-02-10T08:24:00.788830+0000","last_undegraded":"2024-02-10T08:24:00.788830+0000","last_fullsized":"2024-02-10T08:24:00.788830+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790551+0000","last_change":"2024-02-10T08:22:41.910122+0000","last_active":"2024-02-10T08:24:00.790551+0000","last_peered":"2024-02-10T08:24:00.790551+0000","last_clean":"2024-02-10T08:24:00.790551+0000","last_became_active":"2024-02-10T08:22:41.909554+0000","last_became_peered":"2024-02-10T08:22:41.909554+0000","last_unstale":"2024-02-10T08:24:00.790551+0000","last_undegraded":"2024-02-10T08:24:00.790551+0000","last_fullsized":"2024-02-10T08:24:00.790551+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802515+0000","last_change":"2024-02-10T08:22:39.862832+0000","last_active":"2024-02-10T08:24:00.802515+0000","last_peered":"2024-02-10T08:24:00.802515+0000","last_clean":"2024-02-10T08:24:00.802515+0000","last_became_active":"2024-02-10T08:22:39.862382+0000","last_became_peered":"2024-02-10T08:22:39.862382+0000","last_unstale":"2024-02-10T08:24:00.802515+0000","last_undegraded":"2024-02-10T08:24:00.802515+0000","last_fullsized":"2024-02-10T08:24:00.802515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802148+0000","last_change":"2024-02-10T08:22:35.812816+0000","last_active":"2024-02-10T08:24:00.802148+0000","last_peered":"2024-02-10T08:24:00.802148+0000","last_clean":"2024-02-10T08:24:00.802148+0000","last_became_active":"2024-02-10T08:22:35.812330+0000","last_became_peered":"2024-02-10T08:22:35.812330+0000","last_unstale":"2024-02-10T08:24:00.802148+0000","last_undegraded":"2024-02-10T08:24:00.802148+0000","last_fullsized":"2024-02-10T08:24:00.802148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797216+0000","last_change":"2024-02-10T08:22:37.822339+0000","last_active":"2024-02-10T08:24:00.797216+0000","last_peered":"2024-02-10T08:24:00.797216+0000","last_clean":"2024-02-10T08:24:00.797216+0000","last_became_active":"2024-02-10T08:22:37.821789+0000","last_became_peered":"2024-02-10T08:22:37.821789+0000","last_unstale":"2024-02-10T08:24:00.797216+0000","last_undegraded":"2024-02-10T08:24:00.797216+0000","last_fullsized":"2024-02-10T08:24:00.797216+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.800801+0000","last_change":"2024-02-10T08:22:41.917591+0000","last_active":"2024-02-10T08:24:00.800801+0000","last_peered":"2024-02-10T08:24:00.800801+0000","last_clean":"2024-02-10T08:24:00.800801+0000","last_became_active":"2024-02-10T08:22:41.917194+0000","last_became_peered":"2024-02-10T08:22:41.917194+0000","last_unstale":"2024-02-10T08:24:00.800801+0000","last_undegraded":"2024-02-10T08:24:00.800801+0000","last_fullsized":"2024-02-10T08:24:00.800801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788381+0000","last_change":"2024-02-10T08:22:39.858927+0000","last_active":"2024-02-10T08:24:00.788381+0000","last_peered":"2024-02-10T08:24:00.788381+0000","last_clean":"2024-02-10T08:24:00.788381+0000","last_became_active":"2024-02-10T08:22:39.858608+0000","last_became_peered":"2024-02-10T08:22:39.858608+0000","last_unstale":"2024-02-10T08:24:00.788381+0000","last_undegraded":"2024-02-10T08:24:00.788381+0000","last_fullsized":"2024-02-10T08:24:00.788381+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801252+0000","last_change":"2024-02-10T08:22:35.811065+0000","last_active":"2024-02-10T08:24:00.801252+0000","last_peered":"2024-02-10T08:24:00.801252+0000","last_clean":"2024-02-10T08:24:00.801252+0000","last_became_active":"2024-02-10T08:22:35.810215+0000","last_became_peered":"2024-02-10T08:22:35.810215+0000","last_unstale":"2024-02-10T08:24:00.801252+0000","last_undegraded":"2024-02-10T08:24:00.801252+0000","last_fullsized":"2024-02-10T08:24:00.801252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":27,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796826+0000","last_change":"2024-02-10T08:22:37.825421+0000","last_active":"2024-02-10T08:24:00.796826+0000","last_peered":"2024-02-10T08:24:00.796826+0000","last_clean":"2024-02-10T08:24:00.796826+0000","last_became_active":"2024-02-10T08:22:37.823806+0000","last_became_peered":"2024-02-10T08:22:37.823806+0000","last_unstale":"2024-02-10T08:24:00.796826+0000","last_undegraded":"2024-02-10T08:24:00.796826+0000","last_fullsized":"2024-02-10T08:24:00.796826+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796751+0000","last_change":"2024-02-10T08:22:41.912530+0000","last_active":"2024-02-10T08:24:00.796751+0000","last_peered":"2024-02-10T08:24:00.796751+0000","last_clean":"2024-02-10T08:24:00.796751+0000","last_became_active":"2024-02-10T08:22:41.911450+0000","last_became_peered":"2024-02-10T08:22:41.911450+0000","last_unstale":"2024-02-10T08:24:00.796751+0000","last_undegraded":"2024-02-10T08:24:00.796751+0000","last_fullsized":"2024-02-10T08:24:00.796751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796445+0000","last_change":"2024-02-10T08:22:39.853988+0000","last_active":"2024-02-10T08:24:00.796445+0000","last_peered":"2024-02-10T08:24:00.796445+0000","last_clean":"2024-02-10T08:24:00.796445+0000","last_became_active":"2024-02-10T08:22:39.853652+0000","last_became_peered":"2024-02-10T08:22:39.853652+0000","last_unstale":"2024-02-10T08:24:00.796445+0000","last_undegraded":"2024-02-10T08:24:00.796445+0000","last_fullsized":"2024-02-10T08:24:00.796445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"53'1","reported_seq":26,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790304+0000","last_change":"2024-02-10T08:22:35.817379+0000","last_active":"2024-02-10T08:24:00.790304+0000","last_peered":"2024-02-10T08:24:00.790304+0000","last_clean":"2024-02-10T08:24:00.790304+0000","last_became_active":"2024-02-10T08:22:35.816839+0000","last_became_peered":"2024-02-10T08:22:35.816839+0000","last_unstale":"2024-02-10T08:24:00.790304+0000","last_undegraded":"2024-02-10T08:24:00.790304+0000","last_fullsized":"2024-02-10T08:24:00.790304+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.794141+0000","last_change":"2024-02-10T08:22:37.824585+0000","last_active":"2024-02-10T08:24:00.794141+0000","last_peered":"2024-02-10T08:24:00.794141+0000","last_clean":"2024-02-10T08:24:00.794141+0000","last_became_active":"2024-02-10T08:22:37.824205+0000","last_became_peered":"2024-02-10T08:22:37.824205+0000","last_unstale":"2024-02-10T08:24:00.794141+0000","last_undegraded":"2024-02-10T08:24:00.794141+0000","last_fullsized":"2024-02-10T08:24:00.794141+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788716+0000","last_change":"2024-02-10T08:22:41.911689+0000","last_active":"2024-02-10T08:24:00.788716+0000","last_peered":"2024-02-10T08:24:00.788716+0000","last_clean":"2024-02-10T08:24:00.788716+0000","last_became_active":"2024-02-10T08:22:41.911353+0000","last_became_peered":"2024-02-10T08:22:41.911353+0000","last_unstale":"2024-02-10T08:24:00.788716+0000","last_undegraded":"2024-02-10T08:24:00.788716+0000","last_fullsized":"2024-02-10T08:24:00.788716+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796408+0000","last_change":"2024-02-10T08:22:39.842143+0000","last_active":"2024-02-10T08:24:00.796408+0000","last_peered":"2024-02-10T08:24:00.796408+0000","last_clean":"2024-02-10T08:24:00.796408+0000","last_became_active":"2024-02-10T08:22:39.841440+0000","last_became_peered":"2024-02-10T08:22:39.841440+0000","last_unstale":"2024-02-10T08:24:00.796408+0000","last_undegraded":"2024-02-10T08:24:00.796408+0000","last_fullsized":"2024-02-10T08:24:00.796408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796666+0000","last_change":"2024-02-10T08:22:35.804088+0000","last_active":"2024-02-10T08:24:00.796666+0000","last_peered":"2024-02-10T08:24:00.796666+0000","last_clean":"2024-02-10T08:24:00.796666+0000","last_became_active":"2024-02-10T08:22:35.803736+0000","last_became_peered":"2024-02-10T08:22:35.803736+0000","last_unstale":"2024-02-10T08:24:00.796666+0000","last_undegraded":"2024-02-10T08:24:00.796666+0000","last_fullsized":"2024-02-10T08:24:00.796666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":40,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787994+0000","last_change":"2024-02-10T08:22:37.828238+0000","last_active":"2024-02-10T08:24:00.787994+0000","last_peered":"2024-02-10T08:24:00.787994+0000","last_clean":"2024-02-10T08:24:00.787994+0000","last_became_active":"2024-02-10T08:22:37.827915+0000","last_became_peered":"2024-02-10T08:22:37.827915+0000","last_unstale":"2024-02-10T08:24:00.787994+0000","last_undegraded":"2024-02-10T08:24:00.787994+0000","last_fullsized":"2024-02-10T08:24:00.787994+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789327+0000","last_change":"2024-02-10T08:22:41.910634+0000","last_active":"2024-02-10T08:24:00.789327+0000","last_peered":"2024-02-10T08:24:00.789327+0000","last_clean":"2024-02-10T08:24:00.789327+0000","last_became_active":"2024-02-10T08:22:41.910274+0000","last_became_peered":"2024-02-10T08:22:41.910274+0000","last_unstale":"2024-02-10T08:24:00.789327+0000","last_undegraded":"2024-02-10T08:24:00.789327+0000","last_fullsized":"2024-02-10T08:24:00.789327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":80,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.532191+0000","last_change":"2024-02-10T08:22:39.858539+0000","last_active":"2024-02-10T08:27:04.532191+0000","last_peered":"2024-02-10T08:27:04.532191+0000","last_clean":"2024-02-10T08:27:04.532191+0000","last_became_active":"2024-02-10T08:22:39.857353+0000","last_became_peered":"2024-02-10T08:22:39.857353+0000","last_unstale":"2024-02-10T08:27:04.532191+0000","last_undegraded":"2024-02-10T08:27:04.532191+0000","last_fullsized":"2024-02-10T08:27:04.532191+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795260+0000","last_change":"2024-02-10T08:22:35.801699+0000","last_active":"2024-02-10T08:24:00.795260+0000","last_peered":"2024-02-10T08:24:00.795260+0000","last_clean":"2024-02-10T08:24:00.795260+0000","last_became_active":"2024-02-10T08:22:35.801317+0000","last_became_peered":"2024-02-10T08:22:35.801317+0000","last_unstale":"2024-02-10T08:24:00.795260+0000","last_undegraded":"2024-02-10T08:24:00.795260+0000","last_fullsized":"2024-02-10T08:24:00.795260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791215+0000","last_change":"2024-02-10T08:22:38.138960+0000","last_active":"2024-02-10T08:24:00.791215+0000","last_peered":"2024-02-10T08:24:00.791215+0000","last_clean":"2024-02-10T08:24:00.791215+0000","last_became_active":"2024-02-10T08:22:38.138171+0000","last_became_peered":"2024-02-10T08:22:38.138171+0000","last_unstale":"2024-02-10T08:24:00.791215+0000","last_undegraded":"2024-02-10T08:24:00.791215+0000","last_fullsized":"2024-02-10T08:24:00.791215+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796597+0000","last_change":"2024-02-10T08:22:41.913109+0000","last_active":"2024-02-10T08:24:00.796597+0000","last_peered":"2024-02-10T08:24:00.796597+0000","last_clean":"2024-02-10T08:24:00.796597+0000","last_became_active":"2024-02-10T08:22:41.912767+0000","last_became_peered":"2024-02-10T08:22:41.912767+0000","last_unstale":"2024-02-10T08:24:00.796597+0000","last_undegraded":"2024-02-10T08:24:00.796597+0000","last_fullsized":"2024-02-10T08:24:00.796597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791490+0000","last_change":"2024-02-10T08:22:39.862792+0000","last_active":"2024-02-10T08:24:00.791490+0000","last_peered":"2024-02-10T08:24:00.791490+0000","last_clean":"2024-02-10T08:24:00.791490+0000","last_became_active":"2024-02-10T08:22:39.862464+0000","last_became_peered":"2024-02-10T08:22:39.862464+0000","last_unstale":"2024-02-10T08:24:00.791490+0000","last_undegraded":"2024-02-10T08:24:00.791490+0000","last_fullsized":"2024-02-10T08:24:00.791490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786614+0000","last_change":"2024-02-10T08:22:35.808793+0000","last_active":"2024-02-10T08:24:00.786614+0000","last_peered":"2024-02-10T08:24:00.786614+0000","last_clean":"2024-02-10T08:24:00.786614+0000","last_became_active":"2024-02-10T08:22:35.808507+0000","last_became_peered":"2024-02-10T08:22:35.808507+0000","last_unstale":"2024-02-10T08:24:00.786614+0000","last_undegraded":"2024-02-10T08:24:00.786614+0000","last_fullsized":"2024-02-10T08:24:00.786614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:24:01.815858+0000","last_change":"2024-02-10T08:24:01.815858+0000","last_active":"2024-02-10T08:24:01.815858+0000","last_peered":"2024-02-10T08:24:01.815858+0000","last_clean":"2024-02-10T08:24:01.815858+0000","last_became_active":"2024-02-10T08:24:01.799486+0000","last_became_peered":"2024-02-10T08:24:01.799486+0000","last_unstale":"2024-02-10T08:24:01.815858+0000","last_undegraded":"2024-02-10T08:24:01.815858+0000","last_fullsized":"2024-02-10T08:24:01.815858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788165+0000","last_change":"2024-02-10T08:22:41.909799+0000","last_active":"2024-02-10T08:24:00.788165+0000","last_peered":"2024-02-10T08:24:00.788165+0000","last_clean":"2024-02-10T08:24:00.788165+0000","last_became_active":"2024-02-10T08:22:41.909440+0000","last_became_peered":"2024-02-10T08:22:41.909440+0000","last_unstale":"2024-02-10T08:24:00.788165+0000","last_undegraded":"2024-02-10T08:24:00.788165+0000","last_fullsized":"2024-02-10T08:24:00.788165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799419+0000","last_change":"2024-02-10T08:22:39.857163+0000","last_active":"2024-02-10T08:24:00.799419+0000","last_peered":"2024-02-10T08:24:00.799419+0000","last_clean":"2024-02-10T08:24:00.799419+0000","last_became_active":"2024-02-10T08:22:39.856707+0000","last_became_peered":"2024-02-10T08:22:39.856707+0000","last_unstale":"2024-02-10T08:24:00.799419+0000","last_undegraded":"2024-02-10T08:24:00.799419+0000","last_fullsized":"2024-02-10T08:24:00.799419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790849+0000","last_change":"2024-02-10T08:22:35.815708+0000","last_active":"2024-02-10T08:24:00.790849+0000","last_peered":"2024-02-10T08:24:00.790849+0000","last_clean":"2024-02-10T08:24:00.790849+0000","last_became_active":"2024-02-10T08:22:35.815357+0000","last_became_peered":"2024-02-10T08:22:35.815357+0000","last_unstale":"2024-02-10T08:24:00.790849+0000","last_undegraded":"2024-02-10T08:24:00.790849+0000","last_fullsized":"2024-02-10T08:24:00.790849+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786756+0000","last_change":"2024-02-10T08:22:37.826336+0000","last_active":"2024-02-10T08:24:00.786756+0000","last_peered":"2024-02-10T08:24:00.786756+0000","last_clean":"2024-02-10T08:24:00.786756+0000","last_became_active":"2024-02-10T08:22:37.826050+0000","last_became_peered":"2024-02-10T08:22:37.826050+0000","last_unstale":"2024-02-10T08:24:00.786756+0000","last_undegraded":"2024-02-10T08:24:00.786756+0000","last_fullsized":"2024-02-10T08:24:00.786756+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787164+0000","last_change":"2024-02-10T08:22:41.912027+0000","last_active":"2024-02-10T08:24:00.787164+0000","last_peered":"2024-02-10T08:24:00.787164+0000","last_clean":"2024-02-10T08:24:00.787164+0000","last_became_active":"2024-02-10T08:22:41.911148+0000","last_became_peered":"2024-02-10T08:22:41.911148+0000","last_unstale":"2024-02-10T08:24:00.787164+0000","last_undegraded":"2024-02-10T08:24:00.787164+0000","last_fullsized":"2024-02-10T08:24:00.787164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788187+0000","last_change":"2024-02-10T08:22:39.851091+0000","last_active":"2024-02-10T08:24:00.788187+0000","last_peered":"2024-02-10T08:24:00.788187+0000","last_clean":"2024-02-10T08:24:00.788187+0000","last_became_active":"2024-02-10T08:22:39.850737+0000","last_became_peered":"2024-02-10T08:22:39.850737+0000","last_unstale":"2024-02-10T08:24:00.788187+0000","last_undegraded":"2024-02-10T08:24:00.788187+0000","last_fullsized":"2024-02-10T08:24:00.788187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801354+0000","last_change":"2024-02-10T08:22:35.809368+0000","last_active":"2024-02-10T08:24:00.801354+0000","last_peered":"2024-02-10T08:24:00.801354+0000","last_clean":"2024-02-10T08:24:00.801354+0000","last_became_active":"2024-02-10T08:22:35.807887+0000","last_became_peered":"2024-02-10T08:22:35.807887+0000","last_unstale":"2024-02-10T08:24:00.801354+0000","last_undegraded":"2024-02-10T08:24:00.801354+0000","last_fullsized":"2024-02-10T08:24:00.801354+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788100+0000","last_change":"2024-02-10T08:22:38.134858+0000","last_active":"2024-02-10T08:24:00.788100+0000","last_peered":"2024-02-10T08:24:00.788100+0000","last_clean":"2024-02-10T08:24:00.788100+0000","last_became_active":"2024-02-10T08:22:38.134305+0000","last_became_peered":"2024-02-10T08:22:38.134305+0000","last_unstale":"2024-02-10T08:24:00.788100+0000","last_undegraded":"2024-02-10T08:24:00.788100+0000","last_fullsized":"2024-02-10T08:24:00.788100+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789082+0000","last_change":"2024-02-10T08:22:41.917079+0000","last_active":"2024-02-10T08:24:00.789082+0000","last_peered":"2024-02-10T08:24:00.789082+0000","last_clean":"2024-02-10T08:24:00.789082+0000","last_became_active":"2024-02-10T08:22:41.916365+0000","last_became_peered":"2024-02-10T08:22:41.916365+0000","last_unstale":"2024-02-10T08:24:00.789082+0000","last_undegraded":"2024-02-10T08:24:00.789082+0000","last_fullsized":"2024-02-10T08:24:00.789082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787445+0000","last_change":"2024-02-10T08:22:39.855354+0000","last_active":"2024-02-10T08:24:00.787445+0000","last_peered":"2024-02-10T08:24:00.787445+0000","last_clean":"2024-02-10T08:24:00.787445+0000","last_became_active":"2024-02-10T08:22:39.854935+0000","last_became_peered":"2024-02-10T08:22:39.854935+0000","last_unstale":"2024-02-10T08:24:00.787445+0000","last_undegraded":"2024-02-10T08:24:00.787445+0000","last_fullsized":"2024-02-10T08:24:00.787445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788458+0000","last_change":"2024-02-10T08:22:35.805092+0000","last_active":"2024-02-10T08:24:00.788458+0000","last_peered":"2024-02-10T08:24:00.788458+0000","last_clean":"2024-02-10T08:24:00.788458+0000","last_became_active":"2024-02-10T08:22:35.804662+0000","last_became_peered":"2024-02-10T08:22:35.804662+0000","last_unstale":"2024-02-10T08:24:00.788458+0000","last_undegraded":"2024-02-10T08:24:00.788458+0000","last_fullsized":"2024-02-10T08:24:00.788458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795718+0000","last_change":"2024-02-10T08:22:37.830124+0000","last_active":"2024-02-10T08:24:00.795718+0000","last_peered":"2024-02-10T08:24:00.795718+0000","last_clean":"2024-02-10T08:24:00.795718+0000","last_became_active":"2024-02-10T08:22:37.829797+0000","last_became_peered":"2024-02-10T08:22:37.829797+0000","last_unstale":"2024-02-10T08:24:00.795718+0000","last_undegraded":"2024-02-10T08:24:00.795718+0000","last_fullsized":"2024-02-10T08:24:00.795718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786620+0000","last_change":"2024-02-10T08:22:41.913863+0000","last_active":"2024-02-10T08:24:00.786620+0000","last_peered":"2024-02-10T08:24:00.786620+0000","last_clean":"2024-02-10T08:24:00.786620+0000","last_became_active":"2024-02-10T08:22:41.913513+0000","last_became_peered":"2024-02-10T08:22:41.913513+0000","last_unstale":"2024-02-10T08:24:00.786620+0000","last_undegraded":"2024-02-10T08:24:00.786620+0000","last_fullsized":"2024-02-10T08:24:00.786620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795622+0000","last_change":"2024-02-10T08:22:39.857855+0000","last_active":"2024-02-10T08:24:00.795622+0000","last_peered":"2024-02-10T08:24:00.795622+0000","last_clean":"2024-02-10T08:24:00.795622+0000","last_became_active":"2024-02-10T08:22:39.857517+0000","last_became_peered":"2024-02-10T08:22:39.857517+0000","last_unstale":"2024-02-10T08:24:00.795622+0000","last_undegraded":"2024-02-10T08:24:00.795622+0000","last_fullsized":"2024-02-10T08:24:00.795622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796849+0000","last_change":"2024-02-10T08:22:35.809074+0000","last_active":"2024-02-10T08:24:00.796849+0000","last_peered":"2024-02-10T08:24:00.796849+0000","last_clean":"2024-02-10T08:24:00.796849+0000","last_became_active":"2024-02-10T08:22:35.808726+0000","last_became_peered":"2024-02-10T08:22:35.808726+0000","last_unstale":"2024-02-10T08:24:00.796849+0000","last_undegraded":"2024-02-10T08:24:00.796849+0000","last_fullsized":"2024-02-10T08:24:00.796849+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796571+0000","last_change":"2024-02-10T08:22:37.825338+0000","last_active":"2024-02-10T08:24:00.796571+0000","last_peered":"2024-02-10T08:24:00.796571+0000","last_clean":"2024-02-10T08:24:00.796571+0000","last_became_active":"2024-02-10T08:22:37.824235+0000","last_became_peered":"2024-02-10T08:22:37.824235+0000","last_unstale":"2024-02-10T08:24:00.796571+0000","last_undegraded":"2024-02-10T08:24:00.796571+0000","last_fullsized":"2024-02-10T08:24:00.796571+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790903+0000","last_change":"2024-02-10T08:22:41.920436+0000","last_active":"2024-02-10T08:24:00.790903+0000","last_peered":"2024-02-10T08:24:00.790903+0000","last_clean":"2024-02-10T08:24:00.790903+0000","last_became_active":"2024-02-10T08:22:41.920099+0000","last_became_peered":"2024-02-10T08:22:41.920099+0000","last_unstale":"2024-02-10T08:24:00.790903+0000","last_undegraded":"2024-02-10T08:24:00.790903+0000","last_fullsized":"2024-02-10T08:24:00.790903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.532241+0000","last_change":"2024-02-10T08:22:39.856224+0000","last_active":"2024-02-10T08:27:04.532241+0000","last_peered":"2024-02-10T08:27:04.532241+0000","last_clean":"2024-02-10T08:27:04.532241+0000","last_became_active":"2024-02-10T08:22:39.855379+0000","last_became_peered":"2024-02-10T08:22:39.855379+0000","last_unstale":"2024-02-10T08:27:04.532241+0000","last_undegraded":"2024-02-10T08:27:04.532241+0000","last_fullsized":"2024-02-10T08:27:04.532241+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788409+0000","last_change":"2024-02-10T08:22:35.810157+0000","last_active":"2024-02-10T08:24:00.788409+0000","last_peered":"2024-02-10T08:24:00.788409+0000","last_clean":"2024-02-10T08:24:00.788409+0000","last_became_active":"2024-02-10T08:22:35.809855+0000","last_became_peered":"2024-02-10T08:22:35.809855+0000","last_unstale":"2024-02-10T08:24:00.788409+0000","last_undegraded":"2024-02-10T08:24:00.788409+0000","last_fullsized":"2024-02-10T08:24:00.788409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787964+0000","last_change":"2024-02-10T08:22:38.137350+0000","last_active":"2024-02-10T08:24:00.787964+0000","last_peered":"2024-02-10T08:24:00.787964+0000","last_clean":"2024-02-10T08:24:00.787964+0000","last_became_active":"2024-02-10T08:22:38.137010+0000","last_became_peered":"2024-02-10T08:22:38.137010+0000","last_unstale":"2024-02-10T08:24:00.787964+0000","last_undegraded":"2024-02-10T08:24:00.787964+0000","last_fullsized":"2024-02-10T08:24:00.787964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797386+0000","last_change":"2024-02-10T08:22:41.912332+0000","last_active":"2024-02-10T08:24:00.797386+0000","last_peered":"2024-02-10T08:24:00.797386+0000","last_clean":"2024-02-10T08:24:00.797386+0000","last_became_active":"2024-02-10T08:22:41.911934+0000","last_became_peered":"2024-02-10T08:22:41.911934+0000","last_unstale":"2024-02-10T08:24:00.797386+0000","last_undegraded":"2024-02-10T08:24:00.797386+0000","last_fullsized":"2024-02-10T08:24:00.797386+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.531908+0000","last_change":"2024-02-10T08:22:39.849407+0000","last_active":"2024-02-10T08:27:04.531908+0000","last_peered":"2024-02-10T08:27:04.531908+0000","last_clean":"2024-02-10T08:27:04.531908+0000","last_became_active":"2024-02-10T08:22:39.849094+0000","last_became_peered":"2024-02-10T08:22:39.849094+0000","last_unstale":"2024-02-10T08:27:04.531908+0000","last_undegraded":"2024-02-10T08:27:04.531908+0000","last_fullsized":"2024-02-10T08:27:04.531908+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789900+0000","last_change":"2024-02-10T08:22:35.812864+0000","last_active":"2024-02-10T08:24:00.789900+0000","last_peered":"2024-02-10T08:24:00.789900+0000","last_clean":"2024-02-10T08:24:00.789900+0000","last_became_active":"2024-02-10T08:22:35.812469+0000","last_became_peered":"2024-02-10T08:22:35.812469+0000","last_unstale":"2024-02-10T08:24:00.789900+0000","last_undegraded":"2024-02-10T08:24:00.789900+0000","last_fullsized":"2024-02-10T08:24:00.789900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802761+0000","last_change":"2024-02-10T08:22:38.135900+0000","last_active":"2024-02-10T08:24:00.802761+0000","last_peered":"2024-02-10T08:24:00.802761+0000","last_clean":"2024-02-10T08:24:00.802761+0000","last_became_active":"2024-02-10T08:22:38.134547+0000","last_became_peered":"2024-02-10T08:22:38.134547+0000","last_unstale":"2024-02-10T08:24:00.802761+0000","last_undegraded":"2024-02-10T08:24:00.802761+0000","last_fullsized":"2024-02-10T08:24:00.802761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.794347+0000","last_change":"2024-02-10T08:22:41.907068+0000","last_active":"2024-02-10T08:24:00.794347+0000","last_peered":"2024-02-10T08:24:00.794347+0000","last_clean":"2024-02-10T08:24:00.794347+0000","last_became_active":"2024-02-10T08:22:41.906747+0000","last_became_peered":"2024-02-10T08:22:41.906747+0000","last_unstale":"2024-02-10T08:24:00.794347+0000","last_undegraded":"2024-02-10T08:24:00.794347+0000","last_fullsized":"2024-02-10T08:24:00.794347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788582+0000","last_change":"2024-02-10T08:22:39.858620+0000","last_active":"2024-02-10T08:24:00.788582+0000","last_peered":"2024-02-10T08:24:00.788582+0000","last_clean":"2024-02-10T08:24:00.788582+0000","last_became_active":"2024-02-10T08:22:39.858235+0000","last_became_peered":"2024-02-10T08:22:39.858235+0000","last_unstale":"2024-02-10T08:24:00.788582+0000","last_undegraded":"2024-02-10T08:24:00.788582+0000","last_fullsized":"2024-02-10T08:24:00.788582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788077+0000","last_change":"2024-02-10T08:22:35.809976+0000","last_active":"2024-02-10T08:24:00.788077+0000","last_peered":"2024-02-10T08:24:00.788077+0000","last_clean":"2024-02-10T08:24:00.788077+0000","last_became_active":"2024-02-10T08:22:35.809606+0000","last_became_peered":"2024-02-10T08:22:35.809606+0000","last_unstale":"2024-02-10T08:24:00.788077+0000","last_undegraded":"2024-02-10T08:24:00.788077+0000","last_fullsized":"2024-02-10T08:24:00.788077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791317+0000","last_change":"2024-02-10T08:22:38.135981+0000","last_active":"2024-02-10T08:24:00.791317+0000","last_peered":"2024-02-10T08:24:00.791317+0000","last_clean":"2024-02-10T08:24:00.791317+0000","last_became_active":"2024-02-10T08:22:38.134272+0000","last_became_peered":"2024-02-10T08:22:38.134272+0000","last_unstale":"2024-02-10T08:24:00.791317+0000","last_undegraded":"2024-02-10T08:24:00.791317+0000","last_fullsized":"2024-02-10T08:24:00.791317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788328+0000","last_change":"2024-02-10T08:22:41.906356+0000","last_active":"2024-02-10T08:24:00.788328+0000","last_peered":"2024-02-10T08:24:00.788328+0000","last_clean":"2024-02-10T08:24:00.788328+0000","last_became_active":"2024-02-10T08:22:41.905962+0000","last_became_peered":"2024-02-10T08:22:41.905962+0000","last_unstale":"2024-02-10T08:24:00.788328+0000","last_undegraded":"2024-02-10T08:24:00.788328+0000","last_fullsized":"2024-02-10T08:24:00.788328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801557+0000","last_change":"2024-02-10T08:22:39.856962+0000","last_active":"2024-02-10T08:24:00.801557+0000","last_peered":"2024-02-10T08:24:00.801557+0000","last_clean":"2024-02-10T08:24:00.801557+0000","last_became_active":"2024-02-10T08:22:39.856601+0000","last_became_peered":"2024-02-10T08:22:39.856601+0000","last_unstale":"2024-02-10T08:24:00.801557+0000","last_undegraded":"2024-02-10T08:24:00.801557+0000","last_fullsized":"2024-02-10T08:24:00.801557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799027+0000","last_change":"2024-02-10T08:22:35.806905+0000","last_active":"2024-02-10T08:24:00.799027+0000","last_peered":"2024-02-10T08:24:00.799027+0000","last_clean":"2024-02-10T08:24:00.799027+0000","last_became_active":"2024-02-10T08:22:35.806496+0000","last_became_peered":"2024-02-10T08:22:35.806496+0000","last_unstale":"2024-02-10T08:24:00.799027+0000","last_undegraded":"2024-02-10T08:24:00.799027+0000","last_fullsized":"2024-02-10T08:24:00.799027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":26,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788841+0000","last_change":"2024-02-10T08:22:38.138298+0000","last_active":"2024-02-10T08:24:00.788841+0000","last_peered":"2024-02-10T08:24:00.788841+0000","last_clean":"2024-02-10T08:24:00.788841+0000","last_became_active":"2024-02-10T08:22:38.137945+0000","last_became_peered":"2024-02-10T08:22:38.137945+0000","last_unstale":"2024-02-10T08:24:00.788841+0000","last_undegraded":"2024-02-10T08:24:00.788841+0000","last_fullsized":"2024-02-10T08:24:00.788841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803052+0000","last_change":"2024-02-10T08:22:41.916681+0000","last_active":"2024-02-10T08:24:00.803052+0000","last_peered":"2024-02-10T08:24:00.803052+0000","last_clean":"2024-02-10T08:24:00.803052+0000","last_became_active":"2024-02-10T08:22:41.916352+0000","last_became_peered":"2024-02-10T08:22:41.916352+0000","last_unstale":"2024-02-10T08:24:00.803052+0000","last_undegraded":"2024-02-10T08:24:00.803052+0000","last_fullsized":"2024-02-10T08:24:00.803052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'2","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789084+0000","last_change":"2024-02-10T08:22:35.813277+0000","last_active":"2024-02-10T08:24:00.789084+0000","last_peered":"2024-02-10T08:24:00.789084+0000","last_clean":"2024-02-10T08:24:00.789084+0000","last_became_active":"2024-02-10T08:22:35.812979+0000","last_became_peered":"2024-02-10T08:22:35.812979+0000","last_unstale":"2024-02-10T08:24:00.789084+0000","last_undegraded":"2024-02-10T08:24:00.789084+0000","last_fullsized":"2024-02-10T08:24:00.789084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801929+0000","last_change":"2024-02-10T08:22:38.137923+0000","last_active":"2024-02-10T08:24:00.801929+0000","last_peered":"2024-02-10T08:24:00.801929+0000","last_clean":"2024-02-10T08:24:00.801929+0000","last_became_active":"2024-02-10T08:22:38.137491+0000","last_became_peered":"2024-02-10T08:22:38.137491+0000","last_unstale":"2024-02-10T08:24:00.801929+0000","last_undegraded":"2024-02-10T08:24:00.801929+0000","last_fullsized":"2024-02-10T08:24:00.801929+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.532325+0000","last_change":"2024-02-10T08:22:39.859487+0000","last_active":"2024-02-10T08:27:04.532325+0000","last_peered":"2024-02-10T08:27:04.532325+0000","last_clean":"2024-02-10T08:27:04.532325+0000","last_became_active":"2024-02-10T08:22:39.859175+0000","last_became_peered":"2024-02-10T08:22:39.859175+0000","last_unstale":"2024-02-10T08:27:04.532325+0000","last_undegraded":"2024-02-10T08:27:04.532325+0000","last_fullsized":"2024-02-10T08:27:04.532325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787281+0000","last_change":"2024-02-10T08:22:41.904760+0000","last_active":"2024-02-10T08:24:00.787281+0000","last_peered":"2024-02-10T08:24:00.787281+0000","last_clean":"2024-02-10T08:24:00.787281+0000","last_became_active":"2024-02-10T08:22:41.904400+0000","last_became_peered":"2024-02-10T08:22:41.904400+0000","last_unstale":"2024-02-10T08:24:00.787281+0000","last_undegraded":"2024-02-10T08:24:00.787281+0000","last_fullsized":"2024-02-10T08:24:00.787281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788668+0000","last_change":"2024-02-10T08:22:35.808747+0000","last_active":"2024-02-10T08:24:00.788668+0000","last_peered":"2024-02-10T08:24:00.788668+0000","last_clean":"2024-02-10T08:24:00.788668+0000","last_became_active":"2024-02-10T08:22:35.808408+0000","last_became_peered":"2024-02-10T08:22:35.808408+0000","last_unstale":"2024-02-10T08:24:00.788668+0000","last_undegraded":"2024-02-10T08:24:00.788668+0000","last_fullsized":"2024-02-10T08:24:00.788668+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":40,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796501+0000","last_change":"2024-02-10T08:22:37.824196+0000","last_active":"2024-02-10T08:24:00.796501+0000","last_peered":"2024-02-10T08:24:00.796501+0000","last_clean":"2024-02-10T08:24:00.796501+0000","last_became_active":"2024-02-10T08:22:37.823823+0000","last_became_peered":"2024-02-10T08:22:37.823823+0000","last_unstale":"2024-02-10T08:24:00.796501+0000","last_undegraded":"2024-02-10T08:24:00.796501+0000","last_fullsized":"2024-02-10T08:24:00.796501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.792542+0000","last_change":"2024-02-10T08:22:39.847169+0000","last_active":"2024-02-10T08:24:00.792542+0000","last_peered":"2024-02-10T08:24:00.792542+0000","last_clean":"2024-02-10T08:24:00.792542+0000","last_became_active":"2024-02-10T08:22:39.846711+0000","last_became_peered":"2024-02-10T08:22:39.846711+0000","last_unstale":"2024-02-10T08:24:00.792542+0000","last_undegraded":"2024-02-10T08:24:00.792542+0000","last_fullsized":"2024-02-10T08:24:00.792542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787881+0000","last_change":"2024-02-10T08:22:41.910930+0000","last_active":"2024-02-10T08:24:00.787881+0000","last_peered":"2024-02-10T08:24:00.787881+0000","last_clean":"2024-02-10T08:24:00.787881+0000","last_became_active":"2024-02-10T08:22:41.910590+0000","last_became_peered":"2024-02-10T08:22:41.910590+0000","last_unstale":"2024-02-10T08:24:00.787881+0000","last_undegraded":"2024-02-10T08:24:00.787881+0000","last_fullsized":"2024-02-10T08:24:00.787881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.793300+0000","last_change":"2024-02-10T08:22:39.853040+0000","last_active":"2024-02-10T08:24:00.793300+0000","last_peered":"2024-02-10T08:24:00.793300+0000","last_clean":"2024-02-10T08:24:00.793300+0000","last_became_active":"2024-02-10T08:22:39.852654+0000","last_became_peered":"2024-02-10T08:22:39.852654+0000","last_unstale":"2024-02-10T08:24:00.793300+0000","last_undegraded":"2024-02-10T08:24:00.793300+0000","last_fullsized":"2024-02-10T08:24:00.793300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788648+0000","last_change":"2024-02-10T08:22:35.811231+0000","last_active":"2024-02-10T08:24:00.788648+0000","last_peered":"2024-02-10T08:24:00.788648+0000","last_clean":"2024-02-10T08:24:00.788648+0000","last_became_active":"2024-02-10T08:22:35.810874+0000","last_became_peered":"2024-02-10T08:22:35.810874+0000","last_unstale":"2024-02-10T08:24:00.788648+0000","last_undegraded":"2024-02-10T08:24:00.788648+0000","last_fullsized":"2024-02-10T08:24:00.788648+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799970+0000","last_change":"2024-02-10T08:22:38.134202+0000","last_active":"2024-02-10T08:24:00.799970+0000","last_peered":"2024-02-10T08:24:00.799970+0000","last_clean":"2024-02-10T08:24:00.799970+0000","last_became_active":"2024-02-10T08:22:38.133864+0000","last_became_peered":"2024-02-10T08:22:38.133864+0000","last_unstale":"2024-02-10T08:24:00.799970+0000","last_undegraded":"2024-02-10T08:24:00.799970+0000","last_fullsized":"2024-02-10T08:24:00.799970+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789564+0000","last_change":"2024-02-10T08:22:41.909848+0000","last_active":"2024-02-10T08:24:00.789564+0000","last_peered":"2024-02-10T08:24:00.789564+0000","last_clean":"2024-02-10T08:24:00.789564+0000","last_became_active":"2024-02-10T08:22:41.909510+0000","last_became_peered":"2024-02-10T08:22:41.909510+0000","last_unstale":"2024-02-10T08:24:00.789564+0000","last_undegraded":"2024-02-10T08:24:00.789564+0000","last_fullsized":"2024-02-10T08:24:00.789564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.792924+0000","last_change":"2024-02-10T08:22:39.842821+0000","last_active":"2024-02-10T08:24:00.792924+0000","last_peered":"2024-02-10T08:24:00.792924+0000","last_clean":"2024-02-10T08:24:00.792924+0000","last_became_active":"2024-02-10T08:22:39.842406+0000","last_became_peered":"2024-02-10T08:22:39.842406+0000","last_unstale":"2024-02-10T08:24:00.792924+0000","last_undegraded":"2024-02-10T08:24:00.792924+0000","last_fullsized":"2024-02-10T08:24:00.792924+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801390+0000","last_change":"2024-02-10T08:22:35.811474+0000","last_active":"2024-02-10T08:24:00.801390+0000","last_peered":"2024-02-10T08:24:00.801390+0000","last_clean":"2024-02-10T08:24:00.801390+0000","last_became_active":"2024-02-10T08:22:35.810685+0000","last_became_peered":"2024-02-10T08:22:35.810685+0000","last_unstale":"2024-02-10T08:24:00.801390+0000","last_undegraded":"2024-02-10T08:24:00.801390+0000","last_fullsized":"2024-02-10T08:24:00.801390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790423+0000","last_change":"2024-02-10T08:22:37.831975+0000","last_active":"2024-02-10T08:24:00.790423+0000","last_peered":"2024-02-10T08:24:00.790423+0000","last_clean":"2024-02-10T08:24:00.790423+0000","last_became_active":"2024-02-10T08:22:37.831487+0000","last_became_peered":"2024-02-10T08:22:37.831487+0000","last_unstale":"2024-02-10T08:24:00.790423+0000","last_undegraded":"2024-02-10T08:24:00.790423+0000","last_fullsized":"2024-02-10T08:24:00.790423+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788951+0000","last_change":"2024-02-10T08:22:41.907132+0000","last_active":"2024-02-10T08:24:00.788951+0000","last_peered":"2024-02-10T08:24:00.788951+0000","last_clean":"2024-02-10T08:24:00.788951+0000","last_became_active":"2024-02-10T08:22:41.906404+0000","last_became_peered":"2024-02-10T08:22:41.906404+0000","last_unstale":"2024-02-10T08:24:00.788951+0000","last_undegraded":"2024-02-10T08:24:00.788951+0000","last_fullsized":"2024-02-10T08:24:00.788951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802139+0000","last_change":"2024-02-10T08:22:34.802251+0000","last_active":"2024-02-10T08:24:00.802139+0000","last_peered":"2024-02-10T08:24:00.802139+0000","last_clean":"2024-02-10T08:24:00.802139+0000","last_became_active":"2024-02-10T08:22:34.801848+0000","last_became_peered":"2024-02-10T08:22:34.801848+0000","last_unstale":"2024-02-10T08:24:00.802139+0000","last_undegraded":"2024-02-10T08:24:00.802139+0000","last_fullsized":"2024-02-10T08:24:00.802139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_clean_scrub_stamp":"2024-02-10T08:22:33.333925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796344+0000","last_change":"2024-02-10T08:22:35.808818+0000","last_active":"2024-02-10T08:24:00.796344+0000","last_peered":"2024-02-10T08:24:00.796344+0000","last_clean":"2024-02-10T08:24:00.796344+0000","last_became_active":"2024-02-10T08:22:35.808425+0000","last_became_peered":"2024-02-10T08:22:35.808425+0000","last_unstale":"2024-02-10T08:24:00.796344+0000","last_undegraded":"2024-02-10T08:24:00.796344+0000","last_fullsized":"2024-02-10T08:24:00.796344+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.531879+0000","last_change":"2024-02-10T08:22:37.828432+0000","last_active":"2024-02-10T08:27:04.531879+0000","last_peered":"2024-02-10T08:27:04.531879+0000","last_clean":"2024-02-10T08:27:04.531879+0000","last_became_active":"2024-02-10T08:22:37.828051+0000","last_became_peered":"2024-02-10T08:22:37.828051+0000","last_unstale":"2024-02-10T08:27:04.531879+0000","last_undegraded":"2024-02-10T08:27:04.531879+0000","last_fullsized":"2024-02-10T08:27:04.531879+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790477+0000","last_change":"2024-02-10T08:22:39.853353+0000","last_active":"2024-02-10T08:24:00.790477+0000","last_peered":"2024-02-10T08:24:00.790477+0000","last_clean":"2024-02-10T08:24:00.790477+0000","last_became_active":"2024-02-10T08:22:39.852877+0000","last_became_peered":"2024-02-10T08:22:39.852877+0000","last_unstale":"2024-02-10T08:24:00.790477+0000","last_undegraded":"2024-02-10T08:24:00.790477+0000","last_fullsized":"2024-02-10T08:24:00.790477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788792+0000","last_change":"2024-02-10T08:22:41.910864+0000","last_active":"2024-02-10T08:24:00.788792+0000","last_peered":"2024-02-10T08:24:00.788792+0000","last_clean":"2024-02-10T08:24:00.788792+0000","last_became_active":"2024-02-10T08:22:41.910197+0000","last_became_peered":"2024-02-10T08:22:41.910197+0000","last_unstale":"2024-02-10T08:24:00.788792+0000","last_undegraded":"2024-02-10T08:24:00.788792+0000","last_fullsized":"2024-02-10T08:24:00.788792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795084+0000","last_change":"2024-02-10T08:22:35.802822+0000","last_active":"2024-02-10T08:24:00.795084+0000","last_peered":"2024-02-10T08:24:00.795084+0000","last_clean":"2024-02-10T08:24:00.795084+0000","last_became_active":"2024-02-10T08:22:35.802457+0000","last_became_peered":"2024-02-10T08:22:35.802457+0000","last_unstale":"2024-02-10T08:24:00.795084+0000","last_undegraded":"2024-02-10T08:24:00.795084+0000","last_fullsized":"2024-02-10T08:24:00.795084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.792678+0000","last_change":"2024-02-10T08:22:34.800665+0000","last_active":"2024-02-10T08:24:00.792678+0000","last_peered":"2024-02-10T08:24:00.792678+0000","last_clean":"2024-02-10T08:24:00.792678+0000","last_became_active":"2024-02-10T08:22:34.800245+0000","last_became_peered":"2024-02-10T08:22:34.800245+0000","last_unstale":"2024-02-10T08:24:00.792678+0000","last_undegraded":"2024-02-10T08:24:00.792678+0000","last_fullsized":"2024-02-10T08:24:00.792678+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_clean_scrub_stamp":"2024-02-10T08:22:33.333925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797307+0000","last_change":"2024-02-10T08:22:37.833278+0000","last_active":"2024-02-10T08:24:00.797307+0000","last_peered":"2024-02-10T08:24:00.797307+0000","last_clean":"2024-02-10T08:24:00.797307+0000","last_became_active":"2024-02-10T08:22:37.832956+0000","last_became_peered":"2024-02-10T08:22:37.832956+0000","last_unstale":"2024-02-10T08:24:00.797307+0000","last_undegraded":"2024-02-10T08:24:00.797307+0000","last_fullsized":"2024-02-10T08:24:00.797307+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.793210+0000","last_change":"2024-02-10T08:22:39.853002+0000","last_active":"2024-02-10T08:24:00.793210+0000","last_peered":"2024-02-10T08:24:00.793210+0000","last_clean":"2024-02-10T08:24:00.793210+0000","last_became_active":"2024-02-10T08:22:39.852741+0000","last_became_peered":"2024-02-10T08:22:39.852741+0000","last_unstale":"2024-02-10T08:24:00.793210+0000","last_undegraded":"2024-02-10T08:24:00.793210+0000","last_fullsized":"2024-02-10T08:24:00.793210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803255+0000","last_change":"2024-02-10T08:22:41.916310+0000","last_active":"2024-02-10T08:24:00.803255+0000","last_peered":"2024-02-10T08:24:00.803255+0000","last_clean":"2024-02-10T08:24:00.803255+0000","last_became_active":"2024-02-10T08:22:41.915952+0000","last_became_peered":"2024-02-10T08:22:41.915952+0000","last_unstale":"2024-02-10T08:24:00.803255+0000","last_undegraded":"2024-02-10T08:24:00.803255+0000","last_fullsized":"2024-02-10T08:24:00.803255+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787987+0000","last_change":"2024-02-10T08:22:23.014142+0000","last_active":"2024-02-10T08:24:00.787987+0000","last_peered":"2024-02-10T08:24:00.787987+0000","last_clean":"2024-02-10T08:24:00.787987+0000","last_became_active":"2024-02-10T08:22:23.013157+0000","last_became_peered":"2024-02-10T08:22:23.013157+0000","last_unstale":"2024-02-10T08:24:00.787987+0000","last_undegraded":"2024-02-10T08:24:00.787987+0000","last_fullsized":"2024-02-10T08:24:00.787987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:18:50.050684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:18:50.050684+0000","last_clean_scrub_stamp":"2024-02-10T08:18:50.050684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786882+0000","last_change":"2024-02-10T08:22:35.813732+0000","last_active":"2024-02-10T08:24:00.786882+0000","last_peered":"2024-02-10T08:24:00.786882+0000","last_clean":"2024-02-10T08:24:00.786882+0000","last_became_active":"2024-02-10T08:22:35.813424+0000","last_became_peered":"2024-02-10T08:22:35.813424+0000","last_unstale":"2024-02-10T08:24:00.786882+0000","last_undegraded":"2024-02-10T08:24:00.786882+0000","last_fullsized":"2024-02-10T08:24:00.786882+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801295+0000","last_change":"2024-02-10T08:22:38.134456+0000","last_active":"2024-02-10T08:24:00.801295+0000","last_peered":"2024-02-10T08:24:00.801295+0000","last_clean":"2024-02-10T08:24:00.801295+0000","last_became_active":"2024-02-10T08:22:38.133792+0000","last_became_peered":"2024-02-10T08:22:38.133792+0000","last_unstale":"2024-02-10T08:24:00.801295+0000","last_undegraded":"2024-02-10T08:24:00.801295+0000","last_fullsized":"2024-02-10T08:24:00.801295+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789509+0000","last_change":"2024-02-10T08:22:39.863177+0000","last_active":"2024-02-10T08:24:00.789509+0000","last_peered":"2024-02-10T08:24:00.789509+0000","last_clean":"2024-02-10T08:24:00.789509+0000","last_became_active":"2024-02-10T08:22:39.862835+0000","last_became_peered":"2024-02-10T08:22:39.862835+0000","last_unstale":"2024-02-10T08:24:00.789509+0000","last_undegraded":"2024-02-10T08:24:00.789509+0000","last_fullsized":"2024-02-10T08:24:00.789509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795262+0000","last_change":"2024-02-10T08:22:41.912754+0000","last_active":"2024-02-10T08:24:00.795262+0000","last_peered":"2024-02-10T08:24:00.795262+0000","last_clean":"2024-02-10T08:24:00.795262+0000","last_became_active":"2024-02-10T08:22:41.912414+0000","last_became_peered":"2024-02-10T08:22:41.912414+0000","last_unstale":"2024-02-10T08:24:00.795262+0000","last_undegraded":"2024-02-10T08:24:00.795262+0000","last_fullsized":"2024-02-10T08:24:00.795262+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789743+0000","last_change":"2024-02-10T08:22:35.811301+0000","last_active":"2024-02-10T08:24:00.789743+0000","last_peered":"2024-02-10T08:24:00.789743+0000","last_clean":"2024-02-10T08:24:00.789743+0000","last_became_active":"2024-02-10T08:22:35.810874+0000","last_became_peered":"2024-02-10T08:22:35.810874+0000","last_unstale":"2024-02-10T08:24:00.789743+0000","last_undegraded":"2024-02-10T08:24:00.789743+0000","last_fullsized":"2024-02-10T08:24:00.789743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":290,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:10.245279+0000","last_change":"2024-02-10T08:22:34.800568+0000","last_active":"2024-02-10T08:27:10.245279+0000","last_peered":"2024-02-10T08:27:10.245279+0000","last_clean":"2024-02-10T08:27:10.245279+0000","last_became_active":"2024-02-10T08:22:34.800198+0000","last_became_peered":"2024-02-10T08:22:34.800198+0000","last_unstale":"2024-02-10T08:27:10.245279+0000","last_undegraded":"2024-02-10T08:27:10.245279+0000","last_fullsized":"2024-02-10T08:27:10.245279+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_clean_scrub_stamp":"2024-02-10T08:22:33.333925+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":267,"num_read_kb":262,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":35,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791014+0000","last_change":"2024-02-10T08:22:37.831051+0000","last_active":"2024-02-10T08:24:00.791014+0000","last_peered":"2024-02-10T08:24:00.791014+0000","last_clean":"2024-02-10T08:24:00.791014+0000","last_became_active":"2024-02-10T08:22:37.830737+0000","last_became_peered":"2024-02-10T08:22:37.830737+0000","last_unstale":"2024-02-10T08:24:00.791014+0000","last_undegraded":"2024-02-10T08:24:00.791014+0000","last_fullsized":"2024-02-10T08:24:00.791014+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803483+0000","last_change":"2024-02-10T08:22:39.854621+0000","last_active":"2024-02-10T08:24:00.803483+0000","last_peered":"2024-02-10T08:24:00.803483+0000","last_clean":"2024-02-10T08:24:00.803483+0000","last_became_active":"2024-02-10T08:22:39.854057+0000","last_became_peered":"2024-02-10T08:22:39.854057+0000","last_unstale":"2024-02-10T08:24:00.803483+0000","last_undegraded":"2024-02-10T08:24:00.803483+0000","last_fullsized":"2024-02-10T08:24:00.803483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795761+0000","last_change":"2024-02-10T08:22:41.900958+0000","last_active":"2024-02-10T08:24:00.795761+0000","last_peered":"2024-02-10T08:24:00.795761+0000","last_clean":"2024-02-10T08:24:00.795761+0000","last_became_active":"2024-02-10T08:22:41.900620+0000","last_became_peered":"2024-02-10T08:22:41.900620+0000","last_unstale":"2024-02-10T08:24:00.795761+0000","last_undegraded":"2024-02-10T08:24:00.795761+0000","last_fullsized":"2024-02-10T08:24:00.795761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"53'1","reported_seq":25,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795445+0000","last_change":"2024-02-10T08:22:35.808314+0000","last_active":"2024-02-10T08:24:00.795445+0000","last_peered":"2024-02-10T08:24:00.795445+0000","last_clean":"2024-02-10T08:24:00.795445+0000","last_became_active":"2024-02-10T08:22:35.807980+0000","last_became_peered":"2024-02-10T08:22:35.807980+0000","last_unstale":"2024-02-10T08:24:00.795445+0000","last_undegraded":"2024-02-10T08:24:00.795445+0000","last_fullsized":"2024-02-10T08:24:00.795445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":53,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790237+0000","last_change":"2024-02-10T08:22:38.134900+0000","last_active":"2024-02-10T08:24:00.790237+0000","last_peered":"2024-02-10T08:24:00.790237+0000","last_clean":"2024-02-10T08:24:00.790237+0000","last_became_active":"2024-02-10T08:22:38.134256+0000","last_became_peered":"2024-02-10T08:22:38.134256+0000","last_unstale":"2024-02-10T08:24:00.790237+0000","last_undegraded":"2024-02-10T08:24:00.790237+0000","last_fullsized":"2024-02-10T08:24:00.790237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.800113+0000","last_change":"2024-02-10T08:22:39.853556+0000","last_active":"2024-02-10T08:24:00.800113+0000","last_peered":"2024-02-10T08:24:00.800113+0000","last_clean":"2024-02-10T08:24:00.800113+0000","last_became_active":"2024-02-10T08:22:39.853204+0000","last_became_peered":"2024-02-10T08:22:39.853204+0000","last_unstale":"2024-02-10T08:24:00.800113+0000","last_undegraded":"2024-02-10T08:24:00.800113+0000","last_fullsized":"2024-02-10T08:24:00.800113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789179+0000","last_change":"2024-02-10T08:22:41.912577+0000","last_active":"2024-02-10T08:24:00.789179+0000","last_peered":"2024-02-10T08:24:00.789179+0000","last_clean":"2024-02-10T08:24:00.789179+0000","last_became_active":"2024-02-10T08:22:41.912282+0000","last_became_peered":"2024-02-10T08:22:41.912282+0000","last_unstale":"2024-02-10T08:24:00.789179+0000","last_undegraded":"2024-02-10T08:24:00.789179+0000","last_fullsized":"2024-02-10T08:24:00.789179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802590+0000","last_change":"2024-02-10T08:22:35.811409+0000","last_active":"2024-02-10T08:24:00.802590+0000","last_peered":"2024-02-10T08:24:00.802590+0000","last_clean":"2024-02-10T08:24:00.802590+0000","last_became_active":"2024-02-10T08:22:35.810384+0000","last_became_peered":"2024-02-10T08:22:35.810384+0000","last_unstale":"2024-02-10T08:24:00.802590+0000","last_undegraded":"2024-02-10T08:24:00.802590+0000","last_fullsized":"2024-02-10T08:24:00.802590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795684+0000","last_change":"2024-02-10T08:22:37.833208+0000","last_active":"2024-02-10T08:24:00.795684+0000","last_peered":"2024-02-10T08:24:00.795684+0000","last_clean":"2024-02-10T08:24:00.795684+0000","last_became_active":"2024-02-10T08:22:37.832830+0000","last_became_peered":"2024-02-10T08:22:37.832830+0000","last_unstale":"2024-02-10T08:24:00.795684+0000","last_undegraded":"2024-02-10T08:24:00.795684+0000","last_fullsized":"2024-02-10T08:24:00.795684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.531851+0000","last_change":"2024-02-10T08:22:39.857913+0000","last_active":"2024-02-10T08:27:04.531851+0000","last_peered":"2024-02-10T08:27:04.531851+0000","last_clean":"2024-02-10T08:27:04.531851+0000","last_became_active":"2024-02-10T08:22:39.857586+0000","last_became_peered":"2024-02-10T08:22:39.857586+0000","last_unstale":"2024-02-10T08:27:04.531851+0000","last_undegraded":"2024-02-10T08:27:04.531851+0000","last_fullsized":"2024-02-10T08:27:04.531851+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789046+0000","last_change":"2024-02-10T08:22:41.908659+0000","last_active":"2024-02-10T08:24:00.789046+0000","last_peered":"2024-02-10T08:24:00.789046+0000","last_clean":"2024-02-10T08:24:00.789046+0000","last_became_active":"2024-02-10T08:22:41.908344+0000","last_became_peered":"2024-02-10T08:22:41.908344+0000","last_unstale":"2024-02-10T08:24:00.789046+0000","last_undegraded":"2024-02-10T08:24:00.789046+0000","last_fullsized":"2024-02-10T08:24:00.789046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791111+0000","last_change":"2024-02-10T08:22:35.817072+0000","last_active":"2024-02-10T08:24:00.791111+0000","last_peered":"2024-02-10T08:24:00.791111+0000","last_clean":"2024-02-10T08:24:00.791111+0000","last_became_active":"2024-02-10T08:22:35.816735+0000","last_became_peered":"2024-02-10T08:22:35.816735+0000","last_unstale":"2024-02-10T08:24:00.791111+0000","last_undegraded":"2024-02-10T08:24:00.791111+0000","last_fullsized":"2024-02-10T08:24:00.791111+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796053+0000","last_change":"2024-02-10T08:22:37.817785+0000","last_active":"2024-02-10T08:24:00.796053+0000","last_peered":"2024-02-10T08:24:00.796053+0000","last_clean":"2024-02-10T08:24:00.796053+0000","last_became_active":"2024-02-10T08:22:37.817401+0000","last_became_peered":"2024-02-10T08:22:37.817401+0000","last_unstale":"2024-02-10T08:24:00.796053+0000","last_undegraded":"2024-02-10T08:24:00.796053+0000","last_fullsized":"2024-02-10T08:24:00.796053+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.531434+0000","last_change":"2024-02-10T08:22:39.848618+0000","last_active":"2024-02-10T08:27:04.531434+0000","last_peered":"2024-02-10T08:27:04.531434+0000","last_clean":"2024-02-10T08:27:04.531434+0000","last_became_active":"2024-02-10T08:22:39.848241+0000","last_became_peered":"2024-02-10T08:22:39.848241+0000","last_unstale":"2024-02-10T08:27:04.531434+0000","last_undegraded":"2024-02-10T08:27:04.531434+0000","last_fullsized":"2024-02-10T08:27:04.531434+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796326+0000","last_change":"2024-02-10T08:22:41.910884+0000","last_active":"2024-02-10T08:24:00.796326+0000","last_peered":"2024-02-10T08:24:00.796326+0000","last_clean":"2024-02-10T08:24:00.796326+0000","last_became_active":"2024-02-10T08:22:41.910552+0000","last_became_peered":"2024-02-10T08:22:41.910552+0000","last_unstale":"2024-02-10T08:24:00.796326+0000","last_undegraded":"2024-02-10T08:24:00.796326+0000","last_fullsized":"2024-02-10T08:24:00.796326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788356+0000","last_change":"2024-02-10T08:22:35.810186+0000","last_active":"2024-02-10T08:24:00.788356+0000","last_peered":"2024-02-10T08:24:00.788356+0000","last_clean":"2024-02-10T08:24:00.788356+0000","last_became_active":"2024-02-10T08:22:35.809503+0000","last_became_peered":"2024-02-10T08:22:35.809503+0000","last_unstale":"2024-02-10T08:24:00.788356+0000","last_undegraded":"2024-02-10T08:24:00.788356+0000","last_fullsized":"2024-02-10T08:24:00.788356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787113+0000","last_change":"2024-02-10T08:22:38.135493+0000","last_active":"2024-02-10T08:24:00.787113+0000","last_peered":"2024-02-10T08:24:00.787113+0000","last_clean":"2024-02-10T08:24:00.787113+0000","last_became_active":"2024-02-10T08:22:38.135102+0000","last_became_peered":"2024-02-10T08:22:38.135102+0000","last_unstale":"2024-02-10T08:24:00.787113+0000","last_undegraded":"2024-02-10T08:24:00.787113+0000","last_fullsized":"2024-02-10T08:24:00.787113+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795961+0000","last_change":"2024-02-10T08:22:39.848407+0000","last_active":"2024-02-10T08:24:00.795961+0000","last_peered":"2024-02-10T08:24:00.795961+0000","last_clean":"2024-02-10T08:24:00.795961+0000","last_became_active":"2024-02-10T08:22:39.848078+0000","last_became_peered":"2024-02-10T08:22:39.848078+0000","last_unstale":"2024-02-10T08:24:00.795961+0000","last_undegraded":"2024-02-10T08:24:00.795961+0000","last_fullsized":"2024-02-10T08:24:00.795961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801940+0000","last_change":"2024-02-10T08:22:41.917331+0000","last_active":"2024-02-10T08:24:00.801940+0000","last_peered":"2024-02-10T08:24:00.801940+0000","last_clean":"2024-02-10T08:24:00.801940+0000","last_became_active":"2024-02-10T08:22:41.916987+0000","last_became_peered":"2024-02-10T08:22:41.916987+0000","last_unstale":"2024-02-10T08:24:00.801940+0000","last_undegraded":"2024-02-10T08:24:00.801940+0000","last_fullsized":"2024-02-10T08:24:00.801940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":80,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.532478+0000","last_change":"2024-02-10T08:22:39.853321+0000","last_active":"2024-02-10T08:27:04.532478+0000","last_peered":"2024-02-10T08:27:04.532478+0000","last_clean":"2024-02-10T08:27:04.532478+0000","last_became_active":"2024-02-10T08:22:39.852888+0000","last_became_peered":"2024-02-10T08:22:39.852888+0000","last_unstale":"2024-02-10T08:27:04.532478+0000","last_undegraded":"2024-02-10T08:27:04.532478+0000","last_fullsized":"2024-02-10T08:27:04.532478+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786954+0000","last_change":"2024-02-10T08:22:35.801751+0000","last_active":"2024-02-10T08:24:00.786954+0000","last_peered":"2024-02-10T08:24:00.786954+0000","last_clean":"2024-02-10T08:24:00.786954+0000","last_became_active":"2024-02-10T08:22:35.801114+0000","last_became_peered":"2024-02-10T08:22:35.801114+0000","last_unstale":"2024-02-10T08:24:00.786954+0000","last_undegraded":"2024-02-10T08:24:00.786954+0000","last_fullsized":"2024-02-10T08:24:00.786954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:24:03.002621+0000","last_change":"2024-02-10T08:24:03.002621+0000","last_active":"2024-02-10T08:24:03.002621+0000","last_peered":"2024-02-10T08:24:03.002621+0000","last_clean":"2024-02-10T08:24:03.002621+0000","last_became_active":"2024-02-10T08:24:01.796152+0000","last_became_peered":"2024-02-10T08:24:01.796152+0000","last_unstale":"2024-02-10T08:24:03.002621+0000","last_undegraded":"2024-02-10T08:24:03.002621+0000","last_fullsized":"2024-02-10T08:24:03.002621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787435+0000","last_change":"2024-02-10T08:22:41.915223+0000","last_active":"2024-02-10T08:24:00.787435+0000","last_peered":"2024-02-10T08:24:00.787435+0000","last_clean":"2024-02-10T08:24:00.787435+0000","last_became_active":"2024-02-10T08:22:41.914367+0000","last_became_peered":"2024-02-10T08:22:41.914367+0000","last_unstale":"2024-02-10T08:24:00.787435+0000","last_undegraded":"2024-02-10T08:24:00.787435+0000","last_fullsized":"2024-02-10T08:24:00.787435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803426+0000","last_change":"2024-02-10T08:22:39.853353+0000","last_active":"2024-02-10T08:24:00.803426+0000","last_peered":"2024-02-10T08:24:00.803426+0000","last_clean":"2024-02-10T08:24:00.803426+0000","last_became_active":"2024-02-10T08:22:39.853038+0000","last_became_peered":"2024-02-10T08:22:39.853038+0000","last_unstale":"2024-02-10T08:24:00.803426+0000","last_undegraded":"2024-02-10T08:24:00.803426+0000","last_fullsized":"2024-02-10T08:24:00.803426+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796132+0000","last_change":"2024-02-10T08:22:35.810465+0000","last_active":"2024-02-10T08:24:00.796132+0000","last_peered":"2024-02-10T08:24:00.796132+0000","last_clean":"2024-02-10T08:24:00.796132+0000","last_became_active":"2024-02-10T08:22:35.809917+0000","last_became_peered":"2024-02-10T08:22:35.809917+0000","last_unstale":"2024-02-10T08:24:00.796132+0000","last_undegraded":"2024-02-10T08:24:00.796132+0000","last_fullsized":"2024-02-10T08:24:00.796132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796599+0000","last_change":"2024-02-10T08:22:37.825011+0000","last_active":"2024-02-10T08:24:00.796599+0000","last_peered":"2024-02-10T08:24:00.796599+0000","last_clean":"2024-02-10T08:24:00.796599+0000","last_became_active":"2024-02-10T08:22:37.824643+0000","last_became_peered":"2024-02-10T08:22:37.824643+0000","last_unstale":"2024-02-10T08:24:00.796599+0000","last_undegraded":"2024-02-10T08:24:00.796599+0000","last_fullsized":"2024-02-10T08:24:00.796599+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.793346+0000","last_change":"2024-02-10T08:22:41.907067+0000","last_active":"2024-02-10T08:24:00.793346+0000","last_peered":"2024-02-10T08:24:00.793346+0000","last_clean":"2024-02-10T08:24:00.793346+0000","last_became_active":"2024-02-10T08:22:41.906714+0000","last_became_peered":"2024-02-10T08:22:41.906714+0000","last_unstale":"2024-02-10T08:24:00.793346+0000","last_undegraded":"2024-02-10T08:24:00.793346+0000","last_fullsized":"2024-02-10T08:24:00.793346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795555+0000","last_change":"2024-02-10T08:22:39.847053+0000","last_active":"2024-02-10T08:24:00.795555+0000","last_peered":"2024-02-10T08:24:00.795555+0000","last_clean":"2024-02-10T08:24:00.795555+0000","last_became_active":"2024-02-10T08:22:39.846615+0000","last_became_peered":"2024-02-10T08:22:39.846615+0000","last_unstale":"2024-02-10T08:24:00.795555+0000","last_undegraded":"2024-02-10T08:24:00.795555+0000","last_fullsized":"2024-02-10T08:24:00.795555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799717+0000","last_change":"2024-02-10T08:22:35.803370+0000","last_active":"2024-02-10T08:24:00.799717+0000","last_peered":"2024-02-10T08:24:00.799717+0000","last_clean":"2024-02-10T08:24:00.799717+0000","last_became_active":"2024-02-10T08:22:35.802904+0000","last_became_peered":"2024-02-10T08:22:35.802904+0000","last_unstale":"2024-02-10T08:24:00.799717+0000","last_undegraded":"2024-02-10T08:24:00.799717+0000","last_fullsized":"2024-02-10T08:24:00.799717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:24:01.808888+0000","last_change":"2024-02-10T08:24:01.808888+0000","last_active":"2024-02-10T08:24:01.808888+0000","last_peered":"2024-02-10T08:24:01.808888+0000","last_clean":"2024-02-10T08:24:01.808888+0000","last_became_active":"2024-02-10T08:24:01.797575+0000","last_became_peered":"2024-02-10T08:24:01.797575+0000","last_unstale":"2024-02-10T08:24:01.808888+0000","last_undegraded":"2024-02-10T08:24:01.808888+0000","last_fullsized":"2024-02-10T08:24:01.808888+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797034+0000","last_change":"2024-02-10T08:22:41.910486+0000","last_active":"2024-02-10T08:24:00.797034+0000","last_peered":"2024-02-10T08:24:00.797034+0000","last_clean":"2024-02-10T08:24:00.797034+0000","last_became_active":"2024-02-10T08:22:41.910065+0000","last_became_peered":"2024-02-10T08:22:41.910065+0000","last_unstale":"2024-02-10T08:24:00.797034+0000","last_undegraded":"2024-02-10T08:24:00.797034+0000","last_fullsized":"2024-02-10T08:24:00.797034+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":77,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:04.531897+0000","last_change":"2024-02-10T08:22:39.858880+0000","last_active":"2024-02-10T08:27:04.531897+0000","last_peered":"2024-02-10T08:27:04.531897+0000","last_clean":"2024-02-10T08:27:04.531897+0000","last_became_active":"2024-02-10T08:22:39.858514+0000","last_became_peered":"2024-02-10T08:22:39.858514+0000","last_unstale":"2024-02-10T08:27:04.531897+0000","last_undegraded":"2024-02-10T08:27:04.531897+0000","last_fullsized":"2024-02-10T08:27:04.531897+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787589+0000","last_change":"2024-02-10T08:22:35.811552+0000","last_active":"2024-02-10T08:24:00.787589+0000","last_peered":"2024-02-10T08:24:00.787589+0000","last_clean":"2024-02-10T08:24:00.787589+0000","last_became_active":"2024-02-10T08:22:35.810643+0000","last_became_peered":"2024-02-10T08:22:35.810643+0000","last_unstale":"2024-02-10T08:24:00.787589+0000","last_undegraded":"2024-02-10T08:24:00.787589+0000","last_fullsized":"2024-02-10T08:24:00.787589+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796530+0000","last_change":"2024-02-10T08:22:37.822439+0000","last_active":"2024-02-10T08:24:00.796530+0000","last_peered":"2024-02-10T08:24:00.796530+0000","last_clean":"2024-02-10T08:24:00.796530+0000","last_became_active":"2024-02-10T08:22:37.822107+0000","last_became_peered":"2024-02-10T08:22:37.822107+0000","last_unstale":"2024-02-10T08:24:00.796530+0000","last_undegraded":"2024-02-10T08:24:00.796530+0000","last_fullsized":"2024-02-10T08:24:00.796530+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":267,"num_read_kb":262,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447368,"statfs":{"total":95995035648,"available":95690104832,"internally_reserved":0,"allocated":704512,"data_stored":289820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.86699999999999999,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.115,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90000000000000002,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2310000000000001,"5min":1.319,"15min":1.319},"last":0.94199999999999995}]},{"osd":1,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.86599999999999999,"15min":0.84399999999999997},"min":{"1min":0.70699999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.996,"5min":0.92900000000000005,"15min":0.90400000000000003},"min":{"1min":0.83499999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.274,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88500000000000001,"15min":0.87},"min":{"1min":0.69799999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1100000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.69199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.351,"15min":1.351},"last":0.92200000000000004}]},{"osd":3,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89000000000000001,"15min":0.86099999999999999},"min":{"1min":0.71999999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2430000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.79800000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.216,"5min":1.284,"15min":1.284},"last":0.82699999999999996}]},{"osd":4,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.84799999999999998,"15min":0.82499999999999996},"min":{"1min":0.65500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.212,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.87,"15min":0.84199999999999997},"min":{"1min":0.748,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1719999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.85699999999999998,"15min":0.81200000000000006},"min":{"1min":0.76400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1919999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.034,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.74099999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.79300000000000004}]},{"osd":6,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.99199999999999999,"15min":0.94399999999999995},"min":{"1min":0.57399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3500000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.872},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.71799999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.3080000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.81100000000000005}]}]},{"osd":6,"up_from":42,"seq":180388626501,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297752,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447400,"statfs":{"total":95995035648,"available":95690137600,"internally_reserved":0,"allocated":671744,"data_stored":287562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.95999999999999996,"15min":0.94499999999999995},"min":{"1min":0.68600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2769999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.2789999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95799999999999996,"15min":0.95199999999999996},"min":{"1min":0.68000000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2689999999999999,"5min":1.95,"15min":1.95},"last":0.70399999999999996}]},{"osd":1,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.73599999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.379,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0089999999999999,"15min":1.002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.272,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.2190000000000001}]},{"osd":2,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.101,"5min":1.135,"15min":1.135},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.88500000000000001},"min":{"1min":0.78000000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.246,"5min":1.28,"15min":1.28},"last":1.0109999999999999}]},{"osd":3,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.042,"15min":1.036},"min":{"1min":0.88200000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3160000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.038},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.93799999999999994,"15min":0.93000000000000005},"min":{"1min":0.70499999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":1.1240000000000001}]},{"osd":4,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86099999999999999,"15min":0.84499999999999997},"min":{"1min":0.64200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.264,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.145}]},{"osd":5,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.878},"min":{"1min":0.66000000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.155,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0580000000000001,"15min":0.998},"min":{"1min":0.79900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.242,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.88600000000000001}]},{"osd":7,"last update":"Sat Feb 10 08:26:25 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.997,"15min":0.94899999999999995},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.4419999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.093},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1839999999999999,"15min":1.1599999999999999},"min":{"1min":0.92200000000000004,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.3440000000000001,"5min":1.605,"15min":1.605},"last":0.97099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447348,"statfs":{"total":95995035648,"available":95690084352,"internally_reserved":0,"allocated":724992,"data_stored":289296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.61599999999999999},"min":{"1min":0.51900000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99299999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.65300000000000002},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.34599999999999997},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.079}]},{"osd":2,"last update":"Sat Feb 10 08:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80500000000000005,"15min":0.68700000000000006},"min":{"1min":0.50800000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.101,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.871},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80900000000000005,"15min":0.68400000000000005},"min":{"1min":0.60699999999999998,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":1.054,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Feb 10 08:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.88200000000000001,"15min":0.82999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.79700000000000004,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99099999999999999,"5min":1.621,"15min":1.621},"last":0.89300000000000002}]},{"osd":4,"last update":"Sat Feb 10 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3220000000000001,"5min":1.859,"15min":1.859},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0760000000000001,"15min":1.01},"min":{"1min":0.76400000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3460000000000001,"5min":1.488,"15min":1.488},"last":1.1080000000000001}]},{"osd":5,"last update":"Sat Feb 10 08:26:10 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98399999999999999,"15min":0.99399999999999999},"min":{"1min":0.70099999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.391,"5min":1.694,"15min":1.694},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0780000000000001,"15min":1.0580000000000001},"min":{"1min":0.87,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.369,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.2}]},{"osd":6,"last update":"Sat Feb 10 08:26:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.139,"15min":1.1799999999999999},"min":{"1min":0.83999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.069,"5min":1.079,"15min":1.1459999999999999},"min":{"1min":0.77800000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5369999999999999,"5min":2.109,"15min":2.109},"last":1.2370000000000001}]},{"osd":7,"last update":"Sat Feb 10 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.149,"15min":1.167},"min":{"1min":0.877,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.478,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.216,"15min":1.252},"min":{"1min":0.80000000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.4339999999999999,"5min":1.659,"15min":1.659},"last":1.1599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447368,"statfs":{"total":95995035648,"available":95690104832,"internally_reserved":0,"allocated":704512,"data_stored":289517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.61799999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.34799999999999998},"max":{"1min":1.1100000000000001,"5min":1.609,"15min":1.609},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.92200000000000004,"5min":2.319,"15min":2.319},"last":0.90100000000000002}]},{"osd":2,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75800000000000001,"15min":0.66400000000000003},"min":{"1min":0.57999999999999996,"5min":0.497,"15min":0.40500000000000003},"max":{"1min":1.2,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.63},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.72599999999999998},"min":{"1min":0.58499999999999996,"5min":0.47399999999999998,"15min":0.34399999999999997},"max":{"1min":1.044,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.745}]},{"osd":3,"last update":"Sat Feb 10 08:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.871,"15min":0.79700000000000004},"min":{"1min":0.55100000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3500000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.873,"15min":0.83599999999999997},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.41499999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":1.081}]},{"osd":4,"last update":"Sat Feb 10 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.026,"15min":0.98799999999999999},"min":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3169999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0569999999999999,"15min":0.97699999999999998},"min":{"1min":0.68200000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3029999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Sat Feb 10 08:27:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.0760000000000001,"15min":1.038},"min":{"1min":0.84399999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5129999999999999,"5min":1.546,"15min":1.546},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.133,"15min":1.071},"min":{"1min":0.98199999999999998,"5min":0.69999999999999996,"15min":0.64200000000000002},"max":{"1min":1.571,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.117}]},{"osd":6,"last update":"Sat Feb 10 08:26:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1479999999999999,"15min":1.1459999999999999},"min":{"1min":0.90100000000000002,"5min":0.754,"15min":0.754},"max":{"1min":1.4990000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.1399999999999999,"15min":1.1539999999999999},"min":{"1min":0.88100000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.548,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.198}]},{"osd":7,"last update":"Sat Feb 10 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1990000000000001,"15min":1.2130000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3109999999999999,"5min":1.885,"15min":1.885},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.226,"15min":1.266},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5820000000000001,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":1.2430000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297768,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447384,"statfs":{"total":95995035648,"available":95690121216,"internally_reserved":0,"allocated":688128,"data_stored":288588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.64200000000000002},"min":{"1min":0.55100000000000005,"5min":0.38700000000000001,"15min":0.36599999999999999},"max":{"1min":1.046,"5min":1.391,"15min":1.391},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80300000000000005,"15min":0.70399999999999996},"min":{"1min":0.47999999999999998,"5min":0.434,"15min":0.40899999999999997},"max":{"1min":0.95799999999999996,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.49199999999999999}]},{"osd":1,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78200000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.143,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80300000000000005,"15min":0.73399999999999999},"min":{"1min":0.63,"5min":0.51600000000000001,"15min":0.48199999999999998},"max":{"1min":0.96799999999999997,"5min":1.661,"15min":1.661},"last":0.96199999999999997}]},{"osd":3,"last update":"Sat Feb 10 08:26:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.85299999999999998,"15min":0.85899999999999999},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.208,"5min":2.496,"15min":2.496},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.86499999999999999,"15min":0.87},"min":{"1min":0.56599999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.141,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Sat Feb 10 08:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0309999999999999,"15min":1.0029999999999999},"min":{"1min":0.79500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.526,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.052,"5min":1.028,"15min":0.97999999999999998},"min":{"1min":0.74299999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.395,"5min":1.504,"15min":1.504},"last":1.0720000000000001}]},{"osd":5,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.075,"15min":1.0640000000000001},"min":{"1min":0.80600000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3360000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.099,"15min":1.093},"min":{"1min":0.76100000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.514,"5min":1.603,"15min":1.698},"last":0.871}]},{"osd":6,"last update":"Sat Feb 10 08:26:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.133,"15min":1.1140000000000001},"min":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.472,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2030000000000001,"15min":1.226},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.708},"last":0.86699999999999999}]},{"osd":7,"last update":"Sat Feb 10 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.91800000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.514,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1870000000000001,"15min":1.1950000000000001},"min":{"1min":0.91000000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.611,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.999}]}]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297792,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447360,"statfs":{"total":95995035648,"available":95690096640,"internally_reserved":0,"allocated":712704,"data_stored":288741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.74199999999999999,"15min":0.68000000000000005},"min":{"1min":0.57499999999999996,"5min":0.47899999999999998,"15min":0.40899999999999997},"max":{"1min":1.042,"5min":1.262,"15min":1.262},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.753},"min":{"1min":0.58599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":1.0800000000000001,"5min":1.524,"15min":1.524},"last":0.72899999999999998}]},{"osd":1,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82699999999999996,"15min":0.76500000000000001},"min":{"1min":0.66900000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.617,"15min":1.617},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80900000000000005,"15min":0.753},"min":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.68799999999999994}]},{"osd":2,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91600000000000004,"15min":0.84899999999999998},"min":{"1min":0.69299999999999995,"5min":0.55900000000000005,"15min":0.49399999999999999},"max":{"1min":1.236,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91700000000000004,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.216,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.85599999999999998}]},{"osd":4,"last update":"Sat Feb 10 08:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0029999999999999,"15min":0.96299999999999997},"min":{"1min":0.77700000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"max":{"1min":1.51,"5min":1.657,"15min":1.657},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.046,"15min":1.01},"min":{"1min":0.78600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.95799999999999996}]},{"osd":5,"last update":"Sat Feb 10 08:27:11 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.73399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3680000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.139,"5min":1.0940000000000001,"15min":1.0760000000000001},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4259999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1040000000000001}]},{"osd":6,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1279999999999999,"15min":1.1240000000000001},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":1.331},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1200000000000001,"15min":1.0940000000000001},"min":{"1min":0.85199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.468,"5min":1.675,"15min":1.675},"last":0.98599999999999999}]},{"osd":7,"last update":"Sat Feb 10 08:26:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.93500000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.1699999999999999,"15min":1.1259999999999999},"min":{"1min":1.0289999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.175}]}]},{"osd":4,"up_from":30,"seq":128849018964,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297788,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447364,"statfs":{"total":95995035648,"available":95690100736,"internally_reserved":0,"allocated":708608,"data_stored":289295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.89000000000000001,"15min":0.88},"min":{"1min":0.53900000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.26,"5min":1.921,"15min":1.921},"last":1.006},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.879,"15min":0.84599999999999997},"min":{"1min":0.73199999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.26,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.127}]},{"osd":1,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89200000000000002,"15min":0.85999999999999999},"min":{"1min":0.69499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.1839999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.93400000000000005,"15min":0.88900000000000001},"min":{"1min":0.74299999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1120000000000001,"5min":1.409,"15min":1.409},"last":1.083}]},{"osd":2,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.621,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1799999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93700000000000006,"15min":0.90600000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.206,"5min":1.397,"15min":1.397},"last":1.25}]},{"osd":3,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.96499999999999997,"15min":0.91500000000000004},"min":{"1min":0.84699999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1839999999999999,"5min":1.968,"15min":1.968},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.056,"5min":0.95699999999999996,"15min":0.94199999999999995},"min":{"1min":0.91300000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2050000000000001,"5min":1.337,"15min":1.337},"last":0.96799999999999997}]},{"osd":5,"last update":"Sat Feb 10 08:26:09 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.80800000000000005,"15min":0.76600000000000001},"min":{"1min":0.68300000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2210000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.92600000000000005,"15min":0.84999999999999998},"min":{"1min":0.875,"5min":0.433,"15min":0.433},"max":{"1min":1.288,"5min":1.528,"15min":1.528},"last":1.1990000000000001}]},{"osd":6,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.94499999999999995,"15min":0.86899999999999999},"min":{"1min":0.73799999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.208,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.22},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.999,"15min":0.95099999999999996},"min":{"1min":0.67800000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.343,"5min":1.839,"15min":1.839},"last":1.3080000000000001}]},{"osd":7,"last update":"Sat Feb 10 08:26:25 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0800000000000001,"15min":1.0469999999999999},"min":{"1min":0.95299999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3520000000000001,"5min":2.117,"15min":2.117},"last":1.036},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0509999999999999,"15min":1.0249999999999999},"min":{"1min":0.74299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2689999999999999,"5min":1.698,"15min":1.698},"last":1.163}]}]},{"osd":5,"up_from":36,"seq":154618822734,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297764,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447388,"statfs":{"total":95995035648,"available":95690125312,"internally_reserved":0,"allocated":684032,"data_stored":288053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.89700000000000002,"15min":0.872},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83599999999999997,"15min":0.85399999999999998},"min":{"1min":0.55400000000000005,"5min":0.375,"15min":0.375},"max":{"1min":1.107,"5min":1.212,"15min":1.212},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Feb 10 08:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.88700000000000001,"15min":0.85199999999999998},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1839999999999999,"5min":1.248,"15min":1.248},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.86699999999999999,"15min":0.86199999999999999},"min":{"1min":0.64600000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1479999999999999,"5min":1.361,"15min":1.361},"last":1.139}]},{"osd":2,"last update":"Sat Feb 10 08:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.88300000000000001,"15min":0.84199999999999997},"min":{"1min":0.77600000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.149,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.93500000000000005,"15min":0.93300000000000005},"min":{"1min":0.753,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.173,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.99099999999999999}]},{"osd":3,"last update":"Sat Feb 10 08:26:08 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.90200000000000002,"15min":0.83999999999999997},"min":{"1min":0.72799999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2250000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.077,"5min":0.98299999999999998,"15min":0.95199999999999996},"min":{"1min":0.71699999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.91200000000000003}]},{"osd":4,"last update":"Sat Feb 10 08:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.79400000000000004,"15min":0.72199999999999998},"min":{"1min":0.70999999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.038},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.84499999999999997,"15min":0.73799999999999999},"min":{"1min":0.73999999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1830000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.069}]},{"osd":6,"last update":"Sat Feb 10 08:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.94199999999999995,"15min":0.85699999999999998},"min":{"1min":0.70599999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.121,"5min":0.96099999999999997,"15min":0.875},"min":{"1min":0.80800000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.371,"5min":1.375,"15min":1.375},"last":1.1619999999999999}]},{"osd":7,"last update":"Sat Feb 10 08:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":0.96499999999999997,"15min":0.88300000000000001},"min":{"1min":0.88800000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.012},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.032,"15min":0.94199999999999995},"min":{"1min":0.74199999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.101}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T08:27:13.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T08:27:13.449 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:13 smithi067 bash[17385]: audit 2024-02-10T08:27:11.775911+0000 mgr.y (mgr.14547) 178 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:13.449 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:13 smithi067 bash[17385]: cluster 2024-02-10T08:27:12.647905+0000 mgr.y (mgr.14547) 179 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:13.450 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:13 smithi067 bash[24488]: audit 2024-02-10T08:27:11.775911+0000 mgr.y (mgr.14547) 178 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:13.450 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:13 smithi067 bash[24488]: cluster 2024-02-10T08:27:12.647905+0000 mgr.y (mgr.14547) 179 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:27:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:27:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:13 smithi134 bash[18722]: audit 2024-02-10T08:27:11.775911+0000 mgr.y (mgr.14547) 178 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:13 smithi134 bash[18722]: cluster 2024-02-10T08:27:12.647905+0000 mgr.y (mgr.14547) 179 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:14 smithi134 bash[18722]: audit 2024-02-10T08:27:12.707599+0000 mgr.y (mgr.14547) 180 : audit [DBG] from='client.24656 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:27:14.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:14 smithi067 bash[17385]: audit 2024-02-10T08:27:12.707599+0000 mgr.y (mgr.14547) 180 : audit [DBG] from='client.24656 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:27:14.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:14 smithi067 bash[24488]: audit 2024-02-10T08:27:12.707599+0000 mgr.y (mgr.14547) 180 : audit [DBG] from='client.24656 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:27:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:15 smithi134 bash[18722]: cluster 2024-02-10T08:27:14.648600+0000 mgr.y (mgr.14547) 181 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:15 smithi067 bash[17385]: cluster 2024-02-10T08:27:14.648600+0000 mgr.y (mgr.14547) 181 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:15.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:15 smithi067 bash[24488]: cluster 2024-02-10T08:27:14.648600+0000 mgr.y (mgr.14547) 181 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:17.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:16 smithi134 bash[18722]: cluster 2024-02-10T08:27:16.650011+0000 mgr.y (mgr.14547) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:16 smithi067 bash[17385]: cluster 2024-02-10T08:27:16.650011+0000 mgr.y (mgr.14547) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:17.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:16 smithi067 bash[24488]: cluster 2024-02-10T08:27:16.650011+0000 mgr.y (mgr.14547) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:27:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:17] "GET /metrics HTTP/1.1" 200 172200 "" "Prometheus/2.33.4" 2024-02-10T08:27:18.225 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:27:19.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:18 smithi134 bash[18722]: cluster 2024-02-10T08:27:18.650860+0000 mgr.y (mgr.14547) 183 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:19.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:18 smithi067 bash[17385]: cluster 2024-02-10T08:27:18.650860+0000 mgr.y (mgr.14547) 183 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:19.082 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:18 smithi067 bash[24488]: cluster 2024-02-10T08:27:18.650860+0000 mgr.y (mgr.14547) 183 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 -- 172.21.15.67:0/2813662111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 msgr2=0x7f28000f7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2813662111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f28000f7e80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f27f0004870 tx=0x7f27f0009cb0).stop 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 -- 172.21.15.67:0/2813662111 shutdown_connections 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2813662111 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f28000f8560 0x7f2800104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2813662111 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28000fdcf0 0x7f28000fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2813662111 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f28000f7e80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.986+0000 7f2807ea5700 1 -- 172.21.15.67:0/2813662111 >> 172.21.15.67:0/2813662111 conn(0x7f28000f3940 msgr2=0x7f28000f5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:20.991 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- 172.21.15.67:0/2813662111 shutdown_connections 2024-02-10T08:27:20.992 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- 172.21.15.67:0/2813662111 wait complete. 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 Processor -- start 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- start start 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28000f8560 0x7f280010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f28000fdcf0 0x7f280010e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f2800108100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28001085f0 con 0x7f28000fe650 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2800108760 con 0x7f28000f8560 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2800108920 con 0x7f28000fdcf0 2024-02-10T08:27:20.993 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f2800108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f2800108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:36640/0 (socket says 172.21.15.67:36640) 2024-02-10T08:27:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 -- 172.21.15.67:0/2771242854 learned_addr learned my addr 172.21.15.67:0/2771242854 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:27:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2805440700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f28000fdcf0 0x7f280010e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:20.994 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2805c41700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28000f8560 0x7f280010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 -- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f28000fdcf0 msgr2=0x7f280010e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f28000fdcf0 0x7f280010e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 -- 172.21.15.67:0/2771242854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28000f8560 msgr2=0x7f280010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28000f8560 0x7f280010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 -- 172.21.15.67:0/2771242854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0005040 con 0x7f28000fe650 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2806442700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f2800108100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f27f800ec20 tx=0x7f27f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8009950 con 0x7f28000fe650 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f27f800f040 con 0x7f28000fe650 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2800108ba0 con 0x7f28000fe650 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8007d20 con 0x7f28000fe650 2024-02-10T08:27:20.995 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.990+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2800114d90 con 0x7f28000fe650 2024-02-10T08:27:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.994+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280005fad0 con 0x7f28000fe650 2024-02-10T08:27:20.996 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.994+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f27f80092e0 con 0x7f28000fe650 2024-02-10T08:27:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.994+0000 7f27f6ffd700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f27ec061340 0x7f27ec063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.994+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f27f807cb80 con 0x7f28000fe650 2024-02-10T08:27:20.997 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.994+0000 7f2805c41700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f27ec061340 0x7f27ec063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:20.998 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:20.994+0000 7f2805c41700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f27ec061340 0x7f27ec063800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f27fc0048b0 tx=0x7f27fc00b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:21.004 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.002+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f27f8049f60 con 0x7f28000fe650 2024-02-10T08:27:21.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:20 smithi134 bash[18722]: cluster 2024-02-10T08:27:20.651622+0000 mgr.y (mgr.14547) 184 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:20 smithi067 bash[17385]: cluster 2024-02-10T08:27:20.651622+0000 mgr.y (mgr.14547) 184 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:21.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:20 smithi067 bash[24488]: cluster 2024-02-10T08:27:20.651622+0000 mgr.y (mgr.14547) 184 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:21.187 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.186+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 --> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28001095e0 con 0x7f27ec061340 2024-02-10T08:27:21.208 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.206+0000 7f27f6ffd700 1 -- 172.21.15.67:0/2771242854 <== mgr.14547 v2:172.21.15.67:6800/4090043441 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355360 (secure 0 0 0) 0x7f28001095e0 con 0x7f27ec061340 2024-02-10T08:27:21.213 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:27:21.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f27ec061340 msgr2=0x7f27ec063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:21.216 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f27ec061340 0x7f27ec063800 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f27fc0048b0 tx=0x7f27fc00b040).stop 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 msgr2=0x7f2800108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f2800108100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f27f800ec20 tx=0x7f27f800b040).stop 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 shutdown_connections 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f28000fe650 0x7f2800108100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f27ec061340 0x7f27ec063800 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28000f8560 0x7f280010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 --2- 172.21.15.67:0/2771242854 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f28000fdcf0 0x7f280010e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 >> 172.21.15.67:0/2771242854 conn(0x7f28000f3940 msgr2=0x7f28000fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 shutdown_connections 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:21.214+0000 7f2807ea5700 1 -- 172.21.15.67:0/2771242854 wait complete. 2024-02-10T08:27:21.217 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-10T08:27:21.959 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-02-10T08:27:20.651169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":192,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":992,"num_read_kb":744,"num_write":428,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":466,"ondisk_log_size":466,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2382236,"kb_used_data":5468,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747578980,"statfs":{"total":767960285184,"available":765520875520,"internally_reserved":0,"allocated":5599232,"data_stored":2310872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005817"},"pg_stats":[{"pgid":"6.1a","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796133+0000","last_change":"2024-02-10T08:22:41.912064+0000","last_active":"2024-02-10T08:24:00.796133+0000","last_peered":"2024-02-10T08:24:00.796133+0000","last_clean":"2024-02-10T08:24:00.796133+0000","last_became_active":"2024-02-10T08:22:41.911671+0000","last_became_peered":"2024-02-10T08:22:41.911671+0000","last_unstale":"2024-02-10T08:24:00.796133+0000","last_undegraded":"2024-02-10T08:24:00.796133+0000","last_fullsized":"2024-02-10T08:24:00.796133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795122+0000","last_change":"2024-02-10T08:22:39.859638+0000","last_active":"2024-02-10T08:24:00.795122+0000","last_peered":"2024-02-10T08:24:00.795122+0000","last_clean":"2024-02-10T08:24:00.795122+0000","last_became_active":"2024-02-10T08:22:39.859323+0000","last_became_peered":"2024-02-10T08:22:39.859323+0000","last_unstale":"2024-02-10T08:24:00.795122+0000","last_undegraded":"2024-02-10T08:24:00.795122+0000","last_fullsized":"2024-02-10T08:24:00.795122+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790931+0000","last_change":"2024-02-10T08:22:35.811855+0000","last_active":"2024-02-10T08:24:00.790931+0000","last_peered":"2024-02-10T08:24:00.790931+0000","last_clean":"2024-02-10T08:24:00.790931+0000","last_became_active":"2024-02-10T08:22:35.811434+0000","last_became_peered":"2024-02-10T08:22:35.811434+0000","last_unstale":"2024-02-10T08:24:00.790931+0000","last_undegraded":"2024-02-10T08:24:00.790931+0000","last_fullsized":"2024-02-10T08:24:00.790931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796226+0000","last_change":"2024-02-10T08:22:37.824213+0000","last_active":"2024-02-10T08:24:00.796226+0000","last_peered":"2024-02-10T08:24:00.796226+0000","last_clean":"2024-02-10T08:24:00.796226+0000","last_became_active":"2024-02-10T08:22:37.823869+0000","last_became_peered":"2024-02-10T08:22:37.823869+0000","last_unstale":"2024-02-10T08:24:00.796226+0000","last_undegraded":"2024-02-10T08:24:00.796226+0000","last_fullsized":"2024-02-10T08:24:00.796226+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788287+0000","last_change":"2024-02-10T08:22:41.903827+0000","last_active":"2024-02-10T08:24:00.788287+0000","last_peered":"2024-02-10T08:24:00.788287+0000","last_clean":"2024-02-10T08:24:00.788287+0000","last_became_active":"2024-02-10T08:22:41.903451+0000","last_became_peered":"2024-02-10T08:22:41.903451+0000","last_unstale":"2024-02-10T08:24:00.788287+0000","last_undegraded":"2024-02-10T08:24:00.788287+0000","last_fullsized":"2024-02-10T08:24:00.788287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795972+0000","last_change":"2024-02-10T08:22:39.848043+0000","last_active":"2024-02-10T08:24:00.795972+0000","last_peered":"2024-02-10T08:24:00.795972+0000","last_clean":"2024-02-10T08:24:00.795972+0000","last_became_active":"2024-02-10T08:22:39.847674+0000","last_became_peered":"2024-02-10T08:22:39.847674+0000","last_unstale":"2024-02-10T08:24:00.795972+0000","last_undegraded":"2024-02-10T08:24:00.795972+0000","last_fullsized":"2024-02-10T08:24:00.795972+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789074+0000","last_change":"2024-02-10T08:22:35.813711+0000","last_active":"2024-02-10T08:24:00.789074+0000","last_peered":"2024-02-10T08:24:00.789074+0000","last_clean":"2024-02-10T08:24:00.789074+0000","last_became_active":"2024-02-10T08:22:35.813335+0000","last_became_peered":"2024-02-10T08:22:35.813335+0000","last_unstale":"2024-02-10T08:24:00.789074+0000","last_undegraded":"2024-02-10T08:24:00.789074+0000","last_fullsized":"2024-02-10T08:24:00.789074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788830+0000","last_change":"2024-02-10T08:22:37.830239+0000","last_active":"2024-02-10T08:24:00.788830+0000","last_peered":"2024-02-10T08:24:00.788830+0000","last_clean":"2024-02-10T08:24:00.788830+0000","last_became_active":"2024-02-10T08:22:37.829986+0000","last_became_peered":"2024-02-10T08:22:37.829986+0000","last_unstale":"2024-02-10T08:24:00.788830+0000","last_undegraded":"2024-02-10T08:24:00.788830+0000","last_fullsized":"2024-02-10T08:24:00.788830+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790551+0000","last_change":"2024-02-10T08:22:41.910122+0000","last_active":"2024-02-10T08:24:00.790551+0000","last_peered":"2024-02-10T08:24:00.790551+0000","last_clean":"2024-02-10T08:24:00.790551+0000","last_became_active":"2024-02-10T08:22:41.909554+0000","last_became_peered":"2024-02-10T08:22:41.909554+0000","last_unstale":"2024-02-10T08:24:00.790551+0000","last_undegraded":"2024-02-10T08:24:00.790551+0000","last_fullsized":"2024-02-10T08:24:00.790551+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802515+0000","last_change":"2024-02-10T08:22:39.862832+0000","last_active":"2024-02-10T08:24:00.802515+0000","last_peered":"2024-02-10T08:24:00.802515+0000","last_clean":"2024-02-10T08:24:00.802515+0000","last_became_active":"2024-02-10T08:22:39.862382+0000","last_became_peered":"2024-02-10T08:22:39.862382+0000","last_unstale":"2024-02-10T08:24:00.802515+0000","last_undegraded":"2024-02-10T08:24:00.802515+0000","last_fullsized":"2024-02-10T08:24:00.802515+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802148+0000","last_change":"2024-02-10T08:22:35.812816+0000","last_active":"2024-02-10T08:24:00.802148+0000","last_peered":"2024-02-10T08:24:00.802148+0000","last_clean":"2024-02-10T08:24:00.802148+0000","last_became_active":"2024-02-10T08:22:35.812330+0000","last_became_peered":"2024-02-10T08:22:35.812330+0000","last_unstale":"2024-02-10T08:24:00.802148+0000","last_undegraded":"2024-02-10T08:24:00.802148+0000","last_fullsized":"2024-02-10T08:24:00.802148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797216+0000","last_change":"2024-02-10T08:22:37.822339+0000","last_active":"2024-02-10T08:24:00.797216+0000","last_peered":"2024-02-10T08:24:00.797216+0000","last_clean":"2024-02-10T08:24:00.797216+0000","last_became_active":"2024-02-10T08:22:37.821789+0000","last_became_peered":"2024-02-10T08:22:37.821789+0000","last_unstale":"2024-02-10T08:24:00.797216+0000","last_undegraded":"2024-02-10T08:24:00.797216+0000","last_fullsized":"2024-02-10T08:24:00.797216+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.800801+0000","last_change":"2024-02-10T08:22:41.917591+0000","last_active":"2024-02-10T08:24:00.800801+0000","last_peered":"2024-02-10T08:24:00.800801+0000","last_clean":"2024-02-10T08:24:00.800801+0000","last_became_active":"2024-02-10T08:22:41.917194+0000","last_became_peered":"2024-02-10T08:22:41.917194+0000","last_unstale":"2024-02-10T08:24:00.800801+0000","last_undegraded":"2024-02-10T08:24:00.800801+0000","last_fullsized":"2024-02-10T08:24:00.800801+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788381+0000","last_change":"2024-02-10T08:22:39.858927+0000","last_active":"2024-02-10T08:24:00.788381+0000","last_peered":"2024-02-10T08:24:00.788381+0000","last_clean":"2024-02-10T08:24:00.788381+0000","last_became_active":"2024-02-10T08:22:39.858608+0000","last_became_peered":"2024-02-10T08:22:39.858608+0000","last_unstale":"2024-02-10T08:24:00.788381+0000","last_undegraded":"2024-02-10T08:24:00.788381+0000","last_fullsized":"2024-02-10T08:24:00.788381+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801252+0000","last_change":"2024-02-10T08:22:35.811065+0000","last_active":"2024-02-10T08:24:00.801252+0000","last_peered":"2024-02-10T08:24:00.801252+0000","last_clean":"2024-02-10T08:24:00.801252+0000","last_became_active":"2024-02-10T08:22:35.810215+0000","last_became_peered":"2024-02-10T08:22:35.810215+0000","last_unstale":"2024-02-10T08:24:00.801252+0000","last_undegraded":"2024-02-10T08:24:00.801252+0000","last_fullsized":"2024-02-10T08:24:00.801252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":27,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796826+0000","last_change":"2024-02-10T08:22:37.825421+0000","last_active":"2024-02-10T08:24:00.796826+0000","last_peered":"2024-02-10T08:24:00.796826+0000","last_clean":"2024-02-10T08:24:00.796826+0000","last_became_active":"2024-02-10T08:22:37.823806+0000","last_became_peered":"2024-02-10T08:22:37.823806+0000","last_unstale":"2024-02-10T08:24:00.796826+0000","last_undegraded":"2024-02-10T08:24:00.796826+0000","last_fullsized":"2024-02-10T08:24:00.796826+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796751+0000","last_change":"2024-02-10T08:22:41.912530+0000","last_active":"2024-02-10T08:24:00.796751+0000","last_peered":"2024-02-10T08:24:00.796751+0000","last_clean":"2024-02-10T08:24:00.796751+0000","last_became_active":"2024-02-10T08:22:41.911450+0000","last_became_peered":"2024-02-10T08:22:41.911450+0000","last_unstale":"2024-02-10T08:24:00.796751+0000","last_undegraded":"2024-02-10T08:24:00.796751+0000","last_fullsized":"2024-02-10T08:24:00.796751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796445+0000","last_change":"2024-02-10T08:22:39.853988+0000","last_active":"2024-02-10T08:24:00.796445+0000","last_peered":"2024-02-10T08:24:00.796445+0000","last_clean":"2024-02-10T08:24:00.796445+0000","last_became_active":"2024-02-10T08:22:39.853652+0000","last_became_peered":"2024-02-10T08:22:39.853652+0000","last_unstale":"2024-02-10T08:24:00.796445+0000","last_undegraded":"2024-02-10T08:24:00.796445+0000","last_fullsized":"2024-02-10T08:24:00.796445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"53'1","reported_seq":26,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790304+0000","last_change":"2024-02-10T08:22:35.817379+0000","last_active":"2024-02-10T08:24:00.790304+0000","last_peered":"2024-02-10T08:24:00.790304+0000","last_clean":"2024-02-10T08:24:00.790304+0000","last_became_active":"2024-02-10T08:22:35.816839+0000","last_became_peered":"2024-02-10T08:22:35.816839+0000","last_unstale":"2024-02-10T08:24:00.790304+0000","last_undegraded":"2024-02-10T08:24:00.790304+0000","last_fullsized":"2024-02-10T08:24:00.790304+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.794141+0000","last_change":"2024-02-10T08:22:37.824585+0000","last_active":"2024-02-10T08:24:00.794141+0000","last_peered":"2024-02-10T08:24:00.794141+0000","last_clean":"2024-02-10T08:24:00.794141+0000","last_became_active":"2024-02-10T08:22:37.824205+0000","last_became_peered":"2024-02-10T08:22:37.824205+0000","last_unstale":"2024-02-10T08:24:00.794141+0000","last_undegraded":"2024-02-10T08:24:00.794141+0000","last_fullsized":"2024-02-10T08:24:00.794141+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788716+0000","last_change":"2024-02-10T08:22:41.911689+0000","last_active":"2024-02-10T08:24:00.788716+0000","last_peered":"2024-02-10T08:24:00.788716+0000","last_clean":"2024-02-10T08:24:00.788716+0000","last_became_active":"2024-02-10T08:22:41.911353+0000","last_became_peered":"2024-02-10T08:22:41.911353+0000","last_unstale":"2024-02-10T08:24:00.788716+0000","last_undegraded":"2024-02-10T08:24:00.788716+0000","last_fullsized":"2024-02-10T08:24:00.788716+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796408+0000","last_change":"2024-02-10T08:22:39.842143+0000","last_active":"2024-02-10T08:24:00.796408+0000","last_peered":"2024-02-10T08:24:00.796408+0000","last_clean":"2024-02-10T08:24:00.796408+0000","last_became_active":"2024-02-10T08:22:39.841440+0000","last_became_peered":"2024-02-10T08:22:39.841440+0000","last_unstale":"2024-02-10T08:24:00.796408+0000","last_undegraded":"2024-02-10T08:24:00.796408+0000","last_fullsized":"2024-02-10T08:24:00.796408+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796666+0000","last_change":"2024-02-10T08:22:35.804088+0000","last_active":"2024-02-10T08:24:00.796666+0000","last_peered":"2024-02-10T08:24:00.796666+0000","last_clean":"2024-02-10T08:24:00.796666+0000","last_became_active":"2024-02-10T08:22:35.803736+0000","last_became_peered":"2024-02-10T08:22:35.803736+0000","last_unstale":"2024-02-10T08:24:00.796666+0000","last_undegraded":"2024-02-10T08:24:00.796666+0000","last_fullsized":"2024-02-10T08:24:00.796666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":40,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787994+0000","last_change":"2024-02-10T08:22:37.828238+0000","last_active":"2024-02-10T08:24:00.787994+0000","last_peered":"2024-02-10T08:24:00.787994+0000","last_clean":"2024-02-10T08:24:00.787994+0000","last_became_active":"2024-02-10T08:22:37.827915+0000","last_became_peered":"2024-02-10T08:22:37.827915+0000","last_unstale":"2024-02-10T08:24:00.787994+0000","last_undegraded":"2024-02-10T08:24:00.787994+0000","last_fullsized":"2024-02-10T08:24:00.787994+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789327+0000","last_change":"2024-02-10T08:22:41.910634+0000","last_active":"2024-02-10T08:24:00.789327+0000","last_peered":"2024-02-10T08:24:00.789327+0000","last_clean":"2024-02-10T08:24:00.789327+0000","last_became_active":"2024-02-10T08:22:41.910274+0000","last_became_peered":"2024-02-10T08:22:41.910274+0000","last_unstale":"2024-02-10T08:24:00.789327+0000","last_undegraded":"2024-02-10T08:24:00.789327+0000","last_fullsized":"2024-02-10T08:24:00.789327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":81,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536395+0000","last_change":"2024-02-10T08:22:39.858539+0000","last_active":"2024-02-10T08:27:09.536395+0000","last_peered":"2024-02-10T08:27:09.536395+0000","last_clean":"2024-02-10T08:27:09.536395+0000","last_became_active":"2024-02-10T08:22:39.857353+0000","last_became_peered":"2024-02-10T08:22:39.857353+0000","last_unstale":"2024-02-10T08:27:09.536395+0000","last_undegraded":"2024-02-10T08:27:09.536395+0000","last_fullsized":"2024-02-10T08:27:09.536395+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795260+0000","last_change":"2024-02-10T08:22:35.801699+0000","last_active":"2024-02-10T08:24:00.795260+0000","last_peered":"2024-02-10T08:24:00.795260+0000","last_clean":"2024-02-10T08:24:00.795260+0000","last_became_active":"2024-02-10T08:22:35.801317+0000","last_became_peered":"2024-02-10T08:22:35.801317+0000","last_unstale":"2024-02-10T08:24:00.795260+0000","last_undegraded":"2024-02-10T08:24:00.795260+0000","last_fullsized":"2024-02-10T08:24:00.795260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791215+0000","last_change":"2024-02-10T08:22:38.138960+0000","last_active":"2024-02-10T08:24:00.791215+0000","last_peered":"2024-02-10T08:24:00.791215+0000","last_clean":"2024-02-10T08:24:00.791215+0000","last_became_active":"2024-02-10T08:22:38.138171+0000","last_became_peered":"2024-02-10T08:22:38.138171+0000","last_unstale":"2024-02-10T08:24:00.791215+0000","last_undegraded":"2024-02-10T08:24:00.791215+0000","last_fullsized":"2024-02-10T08:24:00.791215+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796597+0000","last_change":"2024-02-10T08:22:41.913109+0000","last_active":"2024-02-10T08:24:00.796597+0000","last_peered":"2024-02-10T08:24:00.796597+0000","last_clean":"2024-02-10T08:24:00.796597+0000","last_became_active":"2024-02-10T08:22:41.912767+0000","last_became_peered":"2024-02-10T08:22:41.912767+0000","last_unstale":"2024-02-10T08:24:00.796597+0000","last_undegraded":"2024-02-10T08:24:00.796597+0000","last_fullsized":"2024-02-10T08:24:00.796597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791490+0000","last_change":"2024-02-10T08:22:39.862792+0000","last_active":"2024-02-10T08:24:00.791490+0000","last_peered":"2024-02-10T08:24:00.791490+0000","last_clean":"2024-02-10T08:24:00.791490+0000","last_became_active":"2024-02-10T08:22:39.862464+0000","last_became_peered":"2024-02-10T08:22:39.862464+0000","last_unstale":"2024-02-10T08:24:00.791490+0000","last_undegraded":"2024-02-10T08:24:00.791490+0000","last_fullsized":"2024-02-10T08:24:00.791490+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786614+0000","last_change":"2024-02-10T08:22:35.808793+0000","last_active":"2024-02-10T08:24:00.786614+0000","last_peered":"2024-02-10T08:24:00.786614+0000","last_clean":"2024-02-10T08:24:00.786614+0000","last_became_active":"2024-02-10T08:22:35.808507+0000","last_became_peered":"2024-02-10T08:22:35.808507+0000","last_unstale":"2024-02-10T08:24:00.786614+0000","last_undegraded":"2024-02-10T08:24:00.786614+0000","last_fullsized":"2024-02-10T08:24:00.786614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:24:01.815858+0000","last_change":"2024-02-10T08:24:01.815858+0000","last_active":"2024-02-10T08:24:01.815858+0000","last_peered":"2024-02-10T08:24:01.815858+0000","last_clean":"2024-02-10T08:24:01.815858+0000","last_became_active":"2024-02-10T08:24:01.799486+0000","last_became_peered":"2024-02-10T08:24:01.799486+0000","last_unstale":"2024-02-10T08:24:01.815858+0000","last_undegraded":"2024-02-10T08:24:01.815858+0000","last_fullsized":"2024-02-10T08:24:01.815858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788165+0000","last_change":"2024-02-10T08:22:41.909799+0000","last_active":"2024-02-10T08:24:00.788165+0000","last_peered":"2024-02-10T08:24:00.788165+0000","last_clean":"2024-02-10T08:24:00.788165+0000","last_became_active":"2024-02-10T08:22:41.909440+0000","last_became_peered":"2024-02-10T08:22:41.909440+0000","last_unstale":"2024-02-10T08:24:00.788165+0000","last_undegraded":"2024-02-10T08:24:00.788165+0000","last_fullsized":"2024-02-10T08:24:00.788165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799419+0000","last_change":"2024-02-10T08:22:39.857163+0000","last_active":"2024-02-10T08:24:00.799419+0000","last_peered":"2024-02-10T08:24:00.799419+0000","last_clean":"2024-02-10T08:24:00.799419+0000","last_became_active":"2024-02-10T08:22:39.856707+0000","last_became_peered":"2024-02-10T08:22:39.856707+0000","last_unstale":"2024-02-10T08:24:00.799419+0000","last_undegraded":"2024-02-10T08:24:00.799419+0000","last_fullsized":"2024-02-10T08:24:00.799419+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790849+0000","last_change":"2024-02-10T08:22:35.815708+0000","last_active":"2024-02-10T08:24:00.790849+0000","last_peered":"2024-02-10T08:24:00.790849+0000","last_clean":"2024-02-10T08:24:00.790849+0000","last_became_active":"2024-02-10T08:22:35.815357+0000","last_became_peered":"2024-02-10T08:22:35.815357+0000","last_unstale":"2024-02-10T08:24:00.790849+0000","last_undegraded":"2024-02-10T08:24:00.790849+0000","last_fullsized":"2024-02-10T08:24:00.790849+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786756+0000","last_change":"2024-02-10T08:22:37.826336+0000","last_active":"2024-02-10T08:24:00.786756+0000","last_peered":"2024-02-10T08:24:00.786756+0000","last_clean":"2024-02-10T08:24:00.786756+0000","last_became_active":"2024-02-10T08:22:37.826050+0000","last_became_peered":"2024-02-10T08:22:37.826050+0000","last_unstale":"2024-02-10T08:24:00.786756+0000","last_undegraded":"2024-02-10T08:24:00.786756+0000","last_fullsized":"2024-02-10T08:24:00.786756+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787164+0000","last_change":"2024-02-10T08:22:41.912027+0000","last_active":"2024-02-10T08:24:00.787164+0000","last_peered":"2024-02-10T08:24:00.787164+0000","last_clean":"2024-02-10T08:24:00.787164+0000","last_became_active":"2024-02-10T08:22:41.911148+0000","last_became_peered":"2024-02-10T08:22:41.911148+0000","last_unstale":"2024-02-10T08:24:00.787164+0000","last_undegraded":"2024-02-10T08:24:00.787164+0000","last_fullsized":"2024-02-10T08:24:00.787164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788187+0000","last_change":"2024-02-10T08:22:39.851091+0000","last_active":"2024-02-10T08:24:00.788187+0000","last_peered":"2024-02-10T08:24:00.788187+0000","last_clean":"2024-02-10T08:24:00.788187+0000","last_became_active":"2024-02-10T08:22:39.850737+0000","last_became_peered":"2024-02-10T08:22:39.850737+0000","last_unstale":"2024-02-10T08:24:00.788187+0000","last_undegraded":"2024-02-10T08:24:00.788187+0000","last_fullsized":"2024-02-10T08:24:00.788187+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801354+0000","last_change":"2024-02-10T08:22:35.809368+0000","last_active":"2024-02-10T08:24:00.801354+0000","last_peered":"2024-02-10T08:24:00.801354+0000","last_clean":"2024-02-10T08:24:00.801354+0000","last_became_active":"2024-02-10T08:22:35.807887+0000","last_became_peered":"2024-02-10T08:22:35.807887+0000","last_unstale":"2024-02-10T08:24:00.801354+0000","last_undegraded":"2024-02-10T08:24:00.801354+0000","last_fullsized":"2024-02-10T08:24:00.801354+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788100+0000","last_change":"2024-02-10T08:22:38.134858+0000","last_active":"2024-02-10T08:24:00.788100+0000","last_peered":"2024-02-10T08:24:00.788100+0000","last_clean":"2024-02-10T08:24:00.788100+0000","last_became_active":"2024-02-10T08:22:38.134305+0000","last_became_peered":"2024-02-10T08:22:38.134305+0000","last_unstale":"2024-02-10T08:24:00.788100+0000","last_undegraded":"2024-02-10T08:24:00.788100+0000","last_fullsized":"2024-02-10T08:24:00.788100+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789082+0000","last_change":"2024-02-10T08:22:41.917079+0000","last_active":"2024-02-10T08:24:00.789082+0000","last_peered":"2024-02-10T08:24:00.789082+0000","last_clean":"2024-02-10T08:24:00.789082+0000","last_became_active":"2024-02-10T08:22:41.916365+0000","last_became_peered":"2024-02-10T08:22:41.916365+0000","last_unstale":"2024-02-10T08:24:00.789082+0000","last_undegraded":"2024-02-10T08:24:00.789082+0000","last_fullsized":"2024-02-10T08:24:00.789082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787445+0000","last_change":"2024-02-10T08:22:39.855354+0000","last_active":"2024-02-10T08:24:00.787445+0000","last_peered":"2024-02-10T08:24:00.787445+0000","last_clean":"2024-02-10T08:24:00.787445+0000","last_became_active":"2024-02-10T08:22:39.854935+0000","last_became_peered":"2024-02-10T08:22:39.854935+0000","last_unstale":"2024-02-10T08:24:00.787445+0000","last_undegraded":"2024-02-10T08:24:00.787445+0000","last_fullsized":"2024-02-10T08:24:00.787445+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788458+0000","last_change":"2024-02-10T08:22:35.805092+0000","last_active":"2024-02-10T08:24:00.788458+0000","last_peered":"2024-02-10T08:24:00.788458+0000","last_clean":"2024-02-10T08:24:00.788458+0000","last_became_active":"2024-02-10T08:22:35.804662+0000","last_became_peered":"2024-02-10T08:22:35.804662+0000","last_unstale":"2024-02-10T08:24:00.788458+0000","last_undegraded":"2024-02-10T08:24:00.788458+0000","last_fullsized":"2024-02-10T08:24:00.788458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795718+0000","last_change":"2024-02-10T08:22:37.830124+0000","last_active":"2024-02-10T08:24:00.795718+0000","last_peered":"2024-02-10T08:24:00.795718+0000","last_clean":"2024-02-10T08:24:00.795718+0000","last_became_active":"2024-02-10T08:22:37.829797+0000","last_became_peered":"2024-02-10T08:22:37.829797+0000","last_unstale":"2024-02-10T08:24:00.795718+0000","last_undegraded":"2024-02-10T08:24:00.795718+0000","last_fullsized":"2024-02-10T08:24:00.795718+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786620+0000","last_change":"2024-02-10T08:22:41.913863+0000","last_active":"2024-02-10T08:24:00.786620+0000","last_peered":"2024-02-10T08:24:00.786620+0000","last_clean":"2024-02-10T08:24:00.786620+0000","last_became_active":"2024-02-10T08:22:41.913513+0000","last_became_peered":"2024-02-10T08:22:41.913513+0000","last_unstale":"2024-02-10T08:24:00.786620+0000","last_undegraded":"2024-02-10T08:24:00.786620+0000","last_fullsized":"2024-02-10T08:24:00.786620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795622+0000","last_change":"2024-02-10T08:22:39.857855+0000","last_active":"2024-02-10T08:24:00.795622+0000","last_peered":"2024-02-10T08:24:00.795622+0000","last_clean":"2024-02-10T08:24:00.795622+0000","last_became_active":"2024-02-10T08:22:39.857517+0000","last_became_peered":"2024-02-10T08:22:39.857517+0000","last_unstale":"2024-02-10T08:24:00.795622+0000","last_undegraded":"2024-02-10T08:24:00.795622+0000","last_fullsized":"2024-02-10T08:24:00.795622+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796849+0000","last_change":"2024-02-10T08:22:35.809074+0000","last_active":"2024-02-10T08:24:00.796849+0000","last_peered":"2024-02-10T08:24:00.796849+0000","last_clean":"2024-02-10T08:24:00.796849+0000","last_became_active":"2024-02-10T08:22:35.808726+0000","last_became_peered":"2024-02-10T08:22:35.808726+0000","last_unstale":"2024-02-10T08:24:00.796849+0000","last_undegraded":"2024-02-10T08:24:00.796849+0000","last_fullsized":"2024-02-10T08:24:00.796849+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796571+0000","last_change":"2024-02-10T08:22:37.825338+0000","last_active":"2024-02-10T08:24:00.796571+0000","last_peered":"2024-02-10T08:24:00.796571+0000","last_clean":"2024-02-10T08:24:00.796571+0000","last_became_active":"2024-02-10T08:22:37.824235+0000","last_became_peered":"2024-02-10T08:22:37.824235+0000","last_unstale":"2024-02-10T08:24:00.796571+0000","last_undegraded":"2024-02-10T08:24:00.796571+0000","last_fullsized":"2024-02-10T08:24:00.796571+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790903+0000","last_change":"2024-02-10T08:22:41.920436+0000","last_active":"2024-02-10T08:24:00.790903+0000","last_peered":"2024-02-10T08:24:00.790903+0000","last_clean":"2024-02-10T08:24:00.790903+0000","last_became_active":"2024-02-10T08:22:41.920099+0000","last_became_peered":"2024-02-10T08:22:41.920099+0000","last_unstale":"2024-02-10T08:24:00.790903+0000","last_undegraded":"2024-02-10T08:24:00.790903+0000","last_fullsized":"2024-02-10T08:24:00.790903+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536200+0000","last_change":"2024-02-10T08:22:39.856224+0000","last_active":"2024-02-10T08:27:09.536200+0000","last_peered":"2024-02-10T08:27:09.536200+0000","last_clean":"2024-02-10T08:27:09.536200+0000","last_became_active":"2024-02-10T08:22:39.855379+0000","last_became_peered":"2024-02-10T08:22:39.855379+0000","last_unstale":"2024-02-10T08:27:09.536200+0000","last_undegraded":"2024-02-10T08:27:09.536200+0000","last_fullsized":"2024-02-10T08:27:09.536200+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788409+0000","last_change":"2024-02-10T08:22:35.810157+0000","last_active":"2024-02-10T08:24:00.788409+0000","last_peered":"2024-02-10T08:24:00.788409+0000","last_clean":"2024-02-10T08:24:00.788409+0000","last_became_active":"2024-02-10T08:22:35.809855+0000","last_became_peered":"2024-02-10T08:22:35.809855+0000","last_unstale":"2024-02-10T08:24:00.788409+0000","last_undegraded":"2024-02-10T08:24:00.788409+0000","last_fullsized":"2024-02-10T08:24:00.788409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787964+0000","last_change":"2024-02-10T08:22:38.137350+0000","last_active":"2024-02-10T08:24:00.787964+0000","last_peered":"2024-02-10T08:24:00.787964+0000","last_clean":"2024-02-10T08:24:00.787964+0000","last_became_active":"2024-02-10T08:22:38.137010+0000","last_became_peered":"2024-02-10T08:22:38.137010+0000","last_unstale":"2024-02-10T08:24:00.787964+0000","last_undegraded":"2024-02-10T08:24:00.787964+0000","last_fullsized":"2024-02-10T08:24:00.787964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797386+0000","last_change":"2024-02-10T08:22:41.912332+0000","last_active":"2024-02-10T08:24:00.797386+0000","last_peered":"2024-02-10T08:24:00.797386+0000","last_clean":"2024-02-10T08:24:00.797386+0000","last_became_active":"2024-02-10T08:22:41.911934+0000","last_became_peered":"2024-02-10T08:22:41.911934+0000","last_unstale":"2024-02-10T08:24:00.797386+0000","last_undegraded":"2024-02-10T08:24:00.797386+0000","last_fullsized":"2024-02-10T08:24:00.797386+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536535+0000","last_change":"2024-02-10T08:22:39.849407+0000","last_active":"2024-02-10T08:27:09.536535+0000","last_peered":"2024-02-10T08:27:09.536535+0000","last_clean":"2024-02-10T08:27:09.536535+0000","last_became_active":"2024-02-10T08:22:39.849094+0000","last_became_peered":"2024-02-10T08:22:39.849094+0000","last_unstale":"2024-02-10T08:27:09.536535+0000","last_undegraded":"2024-02-10T08:27:09.536535+0000","last_fullsized":"2024-02-10T08:27:09.536535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789900+0000","last_change":"2024-02-10T08:22:35.812864+0000","last_active":"2024-02-10T08:24:00.789900+0000","last_peered":"2024-02-10T08:24:00.789900+0000","last_clean":"2024-02-10T08:24:00.789900+0000","last_became_active":"2024-02-10T08:22:35.812469+0000","last_became_peered":"2024-02-10T08:22:35.812469+0000","last_unstale":"2024-02-10T08:24:00.789900+0000","last_undegraded":"2024-02-10T08:24:00.789900+0000","last_fullsized":"2024-02-10T08:24:00.789900+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802761+0000","last_change":"2024-02-10T08:22:38.135900+0000","last_active":"2024-02-10T08:24:00.802761+0000","last_peered":"2024-02-10T08:24:00.802761+0000","last_clean":"2024-02-10T08:24:00.802761+0000","last_became_active":"2024-02-10T08:22:38.134547+0000","last_became_peered":"2024-02-10T08:22:38.134547+0000","last_unstale":"2024-02-10T08:24:00.802761+0000","last_undegraded":"2024-02-10T08:24:00.802761+0000","last_fullsized":"2024-02-10T08:24:00.802761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.794347+0000","last_change":"2024-02-10T08:22:41.907068+0000","last_active":"2024-02-10T08:24:00.794347+0000","last_peered":"2024-02-10T08:24:00.794347+0000","last_clean":"2024-02-10T08:24:00.794347+0000","last_became_active":"2024-02-10T08:22:41.906747+0000","last_became_peered":"2024-02-10T08:22:41.906747+0000","last_unstale":"2024-02-10T08:24:00.794347+0000","last_undegraded":"2024-02-10T08:24:00.794347+0000","last_fullsized":"2024-02-10T08:24:00.794347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788582+0000","last_change":"2024-02-10T08:22:39.858620+0000","last_active":"2024-02-10T08:24:00.788582+0000","last_peered":"2024-02-10T08:24:00.788582+0000","last_clean":"2024-02-10T08:24:00.788582+0000","last_became_active":"2024-02-10T08:22:39.858235+0000","last_became_peered":"2024-02-10T08:22:39.858235+0000","last_unstale":"2024-02-10T08:24:00.788582+0000","last_undegraded":"2024-02-10T08:24:00.788582+0000","last_fullsized":"2024-02-10T08:24:00.788582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788077+0000","last_change":"2024-02-10T08:22:35.809976+0000","last_active":"2024-02-10T08:24:00.788077+0000","last_peered":"2024-02-10T08:24:00.788077+0000","last_clean":"2024-02-10T08:24:00.788077+0000","last_became_active":"2024-02-10T08:22:35.809606+0000","last_became_peered":"2024-02-10T08:22:35.809606+0000","last_unstale":"2024-02-10T08:24:00.788077+0000","last_undegraded":"2024-02-10T08:24:00.788077+0000","last_fullsized":"2024-02-10T08:24:00.788077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791317+0000","last_change":"2024-02-10T08:22:38.135981+0000","last_active":"2024-02-10T08:24:00.791317+0000","last_peered":"2024-02-10T08:24:00.791317+0000","last_clean":"2024-02-10T08:24:00.791317+0000","last_became_active":"2024-02-10T08:22:38.134272+0000","last_became_peered":"2024-02-10T08:22:38.134272+0000","last_unstale":"2024-02-10T08:24:00.791317+0000","last_undegraded":"2024-02-10T08:24:00.791317+0000","last_fullsized":"2024-02-10T08:24:00.791317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788328+0000","last_change":"2024-02-10T08:22:41.906356+0000","last_active":"2024-02-10T08:24:00.788328+0000","last_peered":"2024-02-10T08:24:00.788328+0000","last_clean":"2024-02-10T08:24:00.788328+0000","last_became_active":"2024-02-10T08:22:41.905962+0000","last_became_peered":"2024-02-10T08:22:41.905962+0000","last_unstale":"2024-02-10T08:24:00.788328+0000","last_undegraded":"2024-02-10T08:24:00.788328+0000","last_fullsized":"2024-02-10T08:24:00.788328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801557+0000","last_change":"2024-02-10T08:22:39.856962+0000","last_active":"2024-02-10T08:24:00.801557+0000","last_peered":"2024-02-10T08:24:00.801557+0000","last_clean":"2024-02-10T08:24:00.801557+0000","last_became_active":"2024-02-10T08:22:39.856601+0000","last_became_peered":"2024-02-10T08:22:39.856601+0000","last_unstale":"2024-02-10T08:24:00.801557+0000","last_undegraded":"2024-02-10T08:24:00.801557+0000","last_fullsized":"2024-02-10T08:24:00.801557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799027+0000","last_change":"2024-02-10T08:22:35.806905+0000","last_active":"2024-02-10T08:24:00.799027+0000","last_peered":"2024-02-10T08:24:00.799027+0000","last_clean":"2024-02-10T08:24:00.799027+0000","last_became_active":"2024-02-10T08:22:35.806496+0000","last_became_peered":"2024-02-10T08:22:35.806496+0000","last_unstale":"2024-02-10T08:24:00.799027+0000","last_undegraded":"2024-02-10T08:24:00.799027+0000","last_fullsized":"2024-02-10T08:24:00.799027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":26,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788841+0000","last_change":"2024-02-10T08:22:38.138298+0000","last_active":"2024-02-10T08:24:00.788841+0000","last_peered":"2024-02-10T08:24:00.788841+0000","last_clean":"2024-02-10T08:24:00.788841+0000","last_became_active":"2024-02-10T08:22:38.137945+0000","last_became_peered":"2024-02-10T08:22:38.137945+0000","last_unstale":"2024-02-10T08:24:00.788841+0000","last_undegraded":"2024-02-10T08:24:00.788841+0000","last_fullsized":"2024-02-10T08:24:00.788841+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803052+0000","last_change":"2024-02-10T08:22:41.916681+0000","last_active":"2024-02-10T08:24:00.803052+0000","last_peered":"2024-02-10T08:24:00.803052+0000","last_clean":"2024-02-10T08:24:00.803052+0000","last_became_active":"2024-02-10T08:22:41.916352+0000","last_became_peered":"2024-02-10T08:22:41.916352+0000","last_unstale":"2024-02-10T08:24:00.803052+0000","last_undegraded":"2024-02-10T08:24:00.803052+0000","last_fullsized":"2024-02-10T08:24:00.803052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"53'2","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789084+0000","last_change":"2024-02-10T08:22:35.813277+0000","last_active":"2024-02-10T08:24:00.789084+0000","last_peered":"2024-02-10T08:24:00.789084+0000","last_clean":"2024-02-10T08:24:00.789084+0000","last_became_active":"2024-02-10T08:22:35.812979+0000","last_became_peered":"2024-02-10T08:22:35.812979+0000","last_unstale":"2024-02-10T08:24:00.789084+0000","last_undegraded":"2024-02-10T08:24:00.789084+0000","last_fullsized":"2024-02-10T08:24:00.789084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":42,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801929+0000","last_change":"2024-02-10T08:22:38.137923+0000","last_active":"2024-02-10T08:24:00.801929+0000","last_peered":"2024-02-10T08:24:00.801929+0000","last_clean":"2024-02-10T08:24:00.801929+0000","last_became_active":"2024-02-10T08:22:38.137491+0000","last_became_peered":"2024-02-10T08:22:38.137491+0000","last_unstale":"2024-02-10T08:24:00.801929+0000","last_undegraded":"2024-02-10T08:24:00.801929+0000","last_fullsized":"2024-02-10T08:24:00.801929+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536397+0000","last_change":"2024-02-10T08:22:39.859487+0000","last_active":"2024-02-10T08:27:09.536397+0000","last_peered":"2024-02-10T08:27:09.536397+0000","last_clean":"2024-02-10T08:27:09.536397+0000","last_became_active":"2024-02-10T08:22:39.859175+0000","last_became_peered":"2024-02-10T08:22:39.859175+0000","last_unstale":"2024-02-10T08:27:09.536397+0000","last_undegraded":"2024-02-10T08:27:09.536397+0000","last_fullsized":"2024-02-10T08:27:09.536397+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787281+0000","last_change":"2024-02-10T08:22:41.904760+0000","last_active":"2024-02-10T08:24:00.787281+0000","last_peered":"2024-02-10T08:24:00.787281+0000","last_clean":"2024-02-10T08:24:00.787281+0000","last_became_active":"2024-02-10T08:22:41.904400+0000","last_became_peered":"2024-02-10T08:22:41.904400+0000","last_unstale":"2024-02-10T08:24:00.787281+0000","last_undegraded":"2024-02-10T08:24:00.787281+0000","last_fullsized":"2024-02-10T08:24:00.787281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788668+0000","last_change":"2024-02-10T08:22:35.808747+0000","last_active":"2024-02-10T08:24:00.788668+0000","last_peered":"2024-02-10T08:24:00.788668+0000","last_clean":"2024-02-10T08:24:00.788668+0000","last_became_active":"2024-02-10T08:22:35.808408+0000","last_became_peered":"2024-02-10T08:22:35.808408+0000","last_unstale":"2024-02-10T08:24:00.788668+0000","last_undegraded":"2024-02-10T08:24:00.788668+0000","last_fullsized":"2024-02-10T08:24:00.788668+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":40,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796501+0000","last_change":"2024-02-10T08:22:37.824196+0000","last_active":"2024-02-10T08:24:00.796501+0000","last_peered":"2024-02-10T08:24:00.796501+0000","last_clean":"2024-02-10T08:24:00.796501+0000","last_became_active":"2024-02-10T08:22:37.823823+0000","last_became_peered":"2024-02-10T08:22:37.823823+0000","last_unstale":"2024-02-10T08:24:00.796501+0000","last_undegraded":"2024-02-10T08:24:00.796501+0000","last_fullsized":"2024-02-10T08:24:00.796501+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.792542+0000","last_change":"2024-02-10T08:22:39.847169+0000","last_active":"2024-02-10T08:24:00.792542+0000","last_peered":"2024-02-10T08:24:00.792542+0000","last_clean":"2024-02-10T08:24:00.792542+0000","last_became_active":"2024-02-10T08:22:39.846711+0000","last_became_peered":"2024-02-10T08:22:39.846711+0000","last_unstale":"2024-02-10T08:24:00.792542+0000","last_undegraded":"2024-02-10T08:24:00.792542+0000","last_fullsized":"2024-02-10T08:24:00.792542+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787881+0000","last_change":"2024-02-10T08:22:41.910930+0000","last_active":"2024-02-10T08:24:00.787881+0000","last_peered":"2024-02-10T08:24:00.787881+0000","last_clean":"2024-02-10T08:24:00.787881+0000","last_became_active":"2024-02-10T08:22:41.910590+0000","last_became_peered":"2024-02-10T08:22:41.910590+0000","last_unstale":"2024-02-10T08:24:00.787881+0000","last_undegraded":"2024-02-10T08:24:00.787881+0000","last_fullsized":"2024-02-10T08:24:00.787881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.793300+0000","last_change":"2024-02-10T08:22:39.853040+0000","last_active":"2024-02-10T08:24:00.793300+0000","last_peered":"2024-02-10T08:24:00.793300+0000","last_clean":"2024-02-10T08:24:00.793300+0000","last_became_active":"2024-02-10T08:22:39.852654+0000","last_became_peered":"2024-02-10T08:22:39.852654+0000","last_unstale":"2024-02-10T08:24:00.793300+0000","last_undegraded":"2024-02-10T08:24:00.793300+0000","last_fullsized":"2024-02-10T08:24:00.793300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788648+0000","last_change":"2024-02-10T08:22:35.811231+0000","last_active":"2024-02-10T08:24:00.788648+0000","last_peered":"2024-02-10T08:24:00.788648+0000","last_clean":"2024-02-10T08:24:00.788648+0000","last_became_active":"2024-02-10T08:22:35.810874+0000","last_became_peered":"2024-02-10T08:22:35.810874+0000","last_unstale":"2024-02-10T08:24:00.788648+0000","last_undegraded":"2024-02-10T08:24:00.788648+0000","last_fullsized":"2024-02-10T08:24:00.788648+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":48,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799970+0000","last_change":"2024-02-10T08:22:38.134202+0000","last_active":"2024-02-10T08:24:00.799970+0000","last_peered":"2024-02-10T08:24:00.799970+0000","last_clean":"2024-02-10T08:24:00.799970+0000","last_became_active":"2024-02-10T08:22:38.133864+0000","last_became_peered":"2024-02-10T08:22:38.133864+0000","last_unstale":"2024-02-10T08:24:00.799970+0000","last_undegraded":"2024-02-10T08:24:00.799970+0000","last_fullsized":"2024-02-10T08:24:00.799970+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789564+0000","last_change":"2024-02-10T08:22:41.909848+0000","last_active":"2024-02-10T08:24:00.789564+0000","last_peered":"2024-02-10T08:24:00.789564+0000","last_clean":"2024-02-10T08:24:00.789564+0000","last_became_active":"2024-02-10T08:22:41.909510+0000","last_became_peered":"2024-02-10T08:22:41.909510+0000","last_unstale":"2024-02-10T08:24:00.789564+0000","last_undegraded":"2024-02-10T08:24:00.789564+0000","last_fullsized":"2024-02-10T08:24:00.789564+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.792924+0000","last_change":"2024-02-10T08:22:39.842821+0000","last_active":"2024-02-10T08:24:00.792924+0000","last_peered":"2024-02-10T08:24:00.792924+0000","last_clean":"2024-02-10T08:24:00.792924+0000","last_became_active":"2024-02-10T08:22:39.842406+0000","last_became_peered":"2024-02-10T08:22:39.842406+0000","last_unstale":"2024-02-10T08:24:00.792924+0000","last_undegraded":"2024-02-10T08:24:00.792924+0000","last_fullsized":"2024-02-10T08:24:00.792924+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801390+0000","last_change":"2024-02-10T08:22:35.811474+0000","last_active":"2024-02-10T08:24:00.801390+0000","last_peered":"2024-02-10T08:24:00.801390+0000","last_clean":"2024-02-10T08:24:00.801390+0000","last_became_active":"2024-02-10T08:22:35.810685+0000","last_became_peered":"2024-02-10T08:22:35.810685+0000","last_unstale":"2024-02-10T08:24:00.801390+0000","last_undegraded":"2024-02-10T08:24:00.801390+0000","last_fullsized":"2024-02-10T08:24:00.801390+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":33,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790423+0000","last_change":"2024-02-10T08:22:37.831975+0000","last_active":"2024-02-10T08:24:00.790423+0000","last_peered":"2024-02-10T08:24:00.790423+0000","last_clean":"2024-02-10T08:24:00.790423+0000","last_became_active":"2024-02-10T08:22:37.831487+0000","last_became_peered":"2024-02-10T08:22:37.831487+0000","last_unstale":"2024-02-10T08:24:00.790423+0000","last_undegraded":"2024-02-10T08:24:00.790423+0000","last_fullsized":"2024-02-10T08:24:00.790423+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788951+0000","last_change":"2024-02-10T08:22:41.907132+0000","last_active":"2024-02-10T08:24:00.788951+0000","last_peered":"2024-02-10T08:24:00.788951+0000","last_clean":"2024-02-10T08:24:00.788951+0000","last_became_active":"2024-02-10T08:22:41.906404+0000","last_became_peered":"2024-02-10T08:22:41.906404+0000","last_unstale":"2024-02-10T08:24:00.788951+0000","last_undegraded":"2024-02-10T08:24:00.788951+0000","last_fullsized":"2024-02-10T08:24:00.788951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802139+0000","last_change":"2024-02-10T08:22:34.802251+0000","last_active":"2024-02-10T08:24:00.802139+0000","last_peered":"2024-02-10T08:24:00.802139+0000","last_clean":"2024-02-10T08:24:00.802139+0000","last_became_active":"2024-02-10T08:22:34.801848+0000","last_became_peered":"2024-02-10T08:22:34.801848+0000","last_unstale":"2024-02-10T08:24:00.802139+0000","last_undegraded":"2024-02-10T08:24:00.802139+0000","last_fullsized":"2024-02-10T08:24:00.802139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_clean_scrub_stamp":"2024-02-10T08:22:33.333925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796344+0000","last_change":"2024-02-10T08:22:35.808818+0000","last_active":"2024-02-10T08:24:00.796344+0000","last_peered":"2024-02-10T08:24:00.796344+0000","last_clean":"2024-02-10T08:24:00.796344+0000","last_became_active":"2024-02-10T08:22:35.808425+0000","last_became_peered":"2024-02-10T08:22:35.808425+0000","last_unstale":"2024-02-10T08:24:00.796344+0000","last_undegraded":"2024-02-10T08:24:00.796344+0000","last_fullsized":"2024-02-10T08:24:00.796344+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.535878+0000","last_change":"2024-02-10T08:22:37.828432+0000","last_active":"2024-02-10T08:27:09.535878+0000","last_peered":"2024-02-10T08:27:09.535878+0000","last_clean":"2024-02-10T08:27:09.535878+0000","last_became_active":"2024-02-10T08:22:37.828051+0000","last_became_peered":"2024-02-10T08:22:37.828051+0000","last_unstale":"2024-02-10T08:27:09.535878+0000","last_undegraded":"2024-02-10T08:27:09.535878+0000","last_fullsized":"2024-02-10T08:27:09.535878+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":28,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790477+0000","last_change":"2024-02-10T08:22:39.853353+0000","last_active":"2024-02-10T08:24:00.790477+0000","last_peered":"2024-02-10T08:24:00.790477+0000","last_clean":"2024-02-10T08:24:00.790477+0000","last_became_active":"2024-02-10T08:22:39.852877+0000","last_became_peered":"2024-02-10T08:22:39.852877+0000","last_unstale":"2024-02-10T08:24:00.790477+0000","last_undegraded":"2024-02-10T08:24:00.790477+0000","last_fullsized":"2024-02-10T08:24:00.790477+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788792+0000","last_change":"2024-02-10T08:22:41.910864+0000","last_active":"2024-02-10T08:24:00.788792+0000","last_peered":"2024-02-10T08:24:00.788792+0000","last_clean":"2024-02-10T08:24:00.788792+0000","last_became_active":"2024-02-10T08:22:41.910197+0000","last_became_peered":"2024-02-10T08:22:41.910197+0000","last_unstale":"2024-02-10T08:24:00.788792+0000","last_undegraded":"2024-02-10T08:24:00.788792+0000","last_fullsized":"2024-02-10T08:24:00.788792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795084+0000","last_change":"2024-02-10T08:22:35.802822+0000","last_active":"2024-02-10T08:24:00.795084+0000","last_peered":"2024-02-10T08:24:00.795084+0000","last_clean":"2024-02-10T08:24:00.795084+0000","last_became_active":"2024-02-10T08:22:35.802457+0000","last_became_peered":"2024-02-10T08:22:35.802457+0000","last_unstale":"2024-02-10T08:24:00.795084+0000","last_undegraded":"2024-02-10T08:24:00.795084+0000","last_fullsized":"2024-02-10T08:24:00.795084+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.792678+0000","last_change":"2024-02-10T08:22:34.800665+0000","last_active":"2024-02-10T08:24:00.792678+0000","last_peered":"2024-02-10T08:24:00.792678+0000","last_clean":"2024-02-10T08:24:00.792678+0000","last_became_active":"2024-02-10T08:22:34.800245+0000","last_became_peered":"2024-02-10T08:22:34.800245+0000","last_unstale":"2024-02-10T08:24:00.792678+0000","last_undegraded":"2024-02-10T08:24:00.792678+0000","last_fullsized":"2024-02-10T08:24:00.792678+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_clean_scrub_stamp":"2024-02-10T08:22:33.333925+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":44,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797307+0000","last_change":"2024-02-10T08:22:37.833278+0000","last_active":"2024-02-10T08:24:00.797307+0000","last_peered":"2024-02-10T08:24:00.797307+0000","last_clean":"2024-02-10T08:24:00.797307+0000","last_became_active":"2024-02-10T08:22:37.832956+0000","last_became_peered":"2024-02-10T08:22:37.832956+0000","last_unstale":"2024-02-10T08:24:00.797307+0000","last_undegraded":"2024-02-10T08:24:00.797307+0000","last_fullsized":"2024-02-10T08:24:00.797307+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":13,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.793210+0000","last_change":"2024-02-10T08:22:39.853002+0000","last_active":"2024-02-10T08:24:00.793210+0000","last_peered":"2024-02-10T08:24:00.793210+0000","last_clean":"2024-02-10T08:24:00.793210+0000","last_became_active":"2024-02-10T08:22:39.852741+0000","last_became_peered":"2024-02-10T08:22:39.852741+0000","last_unstale":"2024-02-10T08:24:00.793210+0000","last_undegraded":"2024-02-10T08:24:00.793210+0000","last_fullsized":"2024-02-10T08:24:00.793210+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803255+0000","last_change":"2024-02-10T08:22:41.916310+0000","last_active":"2024-02-10T08:24:00.803255+0000","last_peered":"2024-02-10T08:24:00.803255+0000","last_clean":"2024-02-10T08:24:00.803255+0000","last_became_active":"2024-02-10T08:22:41.915952+0000","last_became_peered":"2024-02-10T08:22:41.915952+0000","last_unstale":"2024-02-10T08:24:00.803255+0000","last_undegraded":"2024-02-10T08:24:00.803255+0000","last_fullsized":"2024-02-10T08:24:00.803255+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787987+0000","last_change":"2024-02-10T08:22:23.014142+0000","last_active":"2024-02-10T08:24:00.787987+0000","last_peered":"2024-02-10T08:24:00.787987+0000","last_clean":"2024-02-10T08:24:00.787987+0000","last_became_active":"2024-02-10T08:22:23.013157+0000","last_became_peered":"2024-02-10T08:22:23.013157+0000","last_unstale":"2024-02-10T08:24:00.787987+0000","last_undegraded":"2024-02-10T08:24:00.787987+0000","last_fullsized":"2024-02-10T08:24:00.787987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:18:50.050684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:18:50.050684+0000","last_clean_scrub_stamp":"2024-02-10T08:18:50.050684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786882+0000","last_change":"2024-02-10T08:22:35.813732+0000","last_active":"2024-02-10T08:24:00.786882+0000","last_peered":"2024-02-10T08:24:00.786882+0000","last_clean":"2024-02-10T08:24:00.786882+0000","last_became_active":"2024-02-10T08:22:35.813424+0000","last_became_peered":"2024-02-10T08:22:35.813424+0000","last_unstale":"2024-02-10T08:24:00.786882+0000","last_undegraded":"2024-02-10T08:24:00.786882+0000","last_fullsized":"2024-02-10T08:24:00.786882+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":39,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801295+0000","last_change":"2024-02-10T08:22:38.134456+0000","last_active":"2024-02-10T08:24:00.801295+0000","last_peered":"2024-02-10T08:24:00.801295+0000","last_clean":"2024-02-10T08:24:00.801295+0000","last_became_active":"2024-02-10T08:22:38.133792+0000","last_became_peered":"2024-02-10T08:22:38.133792+0000","last_unstale":"2024-02-10T08:24:00.801295+0000","last_undegraded":"2024-02-10T08:24:00.801295+0000","last_fullsized":"2024-02-10T08:24:00.801295+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789509+0000","last_change":"2024-02-10T08:22:39.863177+0000","last_active":"2024-02-10T08:24:00.789509+0000","last_peered":"2024-02-10T08:24:00.789509+0000","last_clean":"2024-02-10T08:24:00.789509+0000","last_became_active":"2024-02-10T08:22:39.862835+0000","last_became_peered":"2024-02-10T08:22:39.862835+0000","last_unstale":"2024-02-10T08:24:00.789509+0000","last_undegraded":"2024-02-10T08:24:00.789509+0000","last_fullsized":"2024-02-10T08:24:00.789509+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795262+0000","last_change":"2024-02-10T08:22:41.912754+0000","last_active":"2024-02-10T08:24:00.795262+0000","last_peered":"2024-02-10T08:24:00.795262+0000","last_clean":"2024-02-10T08:24:00.795262+0000","last_became_active":"2024-02-10T08:22:41.912414+0000","last_became_peered":"2024-02-10T08:22:41.912414+0000","last_unstale":"2024-02-10T08:24:00.795262+0000","last_undegraded":"2024-02-10T08:24:00.795262+0000","last_fullsized":"2024-02-10T08:24:00.795262+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789743+0000","last_change":"2024-02-10T08:22:35.811301+0000","last_active":"2024-02-10T08:24:00.789743+0000","last_peered":"2024-02-10T08:24:00.789743+0000","last_clean":"2024-02-10T08:24:00.789743+0000","last_became_active":"2024-02-10T08:22:35.810874+0000","last_became_peered":"2024-02-10T08:22:35.810874+0000","last_unstale":"2024-02-10T08:24:00.789743+0000","last_undegraded":"2024-02-10T08:24:00.789743+0000","last_fullsized":"2024-02-10T08:24:00.789743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":295,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:15.256029+0000","last_change":"2024-02-10T08:22:34.800568+0000","last_active":"2024-02-10T08:27:15.256029+0000","last_peered":"2024-02-10T08:27:15.256029+0000","last_clean":"2024-02-10T08:27:15.256029+0000","last_became_active":"2024-02-10T08:22:34.800198+0000","last_became_peered":"2024-02-10T08:22:34.800198+0000","last_unstale":"2024-02-10T08:27:15.256029+0000","last_undegraded":"2024-02-10T08:27:15.256029+0000","last_fullsized":"2024-02-10T08:27:15.256029+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:33.333925+0000","last_clean_scrub_stamp":"2024-02-10T08:22:33.333925+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":272,"num_read_kb":267,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":35,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791014+0000","last_change":"2024-02-10T08:22:37.831051+0000","last_active":"2024-02-10T08:24:00.791014+0000","last_peered":"2024-02-10T08:24:00.791014+0000","last_clean":"2024-02-10T08:24:00.791014+0000","last_became_active":"2024-02-10T08:22:37.830737+0000","last_became_peered":"2024-02-10T08:22:37.830737+0000","last_unstale":"2024-02-10T08:24:00.791014+0000","last_undegraded":"2024-02-10T08:24:00.791014+0000","last_fullsized":"2024-02-10T08:24:00.791014+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803483+0000","last_change":"2024-02-10T08:22:39.854621+0000","last_active":"2024-02-10T08:24:00.803483+0000","last_peered":"2024-02-10T08:24:00.803483+0000","last_clean":"2024-02-10T08:24:00.803483+0000","last_became_active":"2024-02-10T08:22:39.854057+0000","last_became_peered":"2024-02-10T08:22:39.854057+0000","last_unstale":"2024-02-10T08:24:00.803483+0000","last_undegraded":"2024-02-10T08:24:00.803483+0000","last_fullsized":"2024-02-10T08:24:00.803483+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795761+0000","last_change":"2024-02-10T08:22:41.900958+0000","last_active":"2024-02-10T08:24:00.795761+0000","last_peered":"2024-02-10T08:24:00.795761+0000","last_clean":"2024-02-10T08:24:00.795761+0000","last_became_active":"2024-02-10T08:22:41.900620+0000","last_became_peered":"2024-02-10T08:22:41.900620+0000","last_unstale":"2024-02-10T08:24:00.795761+0000","last_undegraded":"2024-02-10T08:24:00.795761+0000","last_fullsized":"2024-02-10T08:24:00.795761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"53'1","reported_seq":25,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795445+0000","last_change":"2024-02-10T08:22:35.808314+0000","last_active":"2024-02-10T08:24:00.795445+0000","last_peered":"2024-02-10T08:24:00.795445+0000","last_clean":"2024-02-10T08:24:00.795445+0000","last_became_active":"2024-02-10T08:22:35.807980+0000","last_became_peered":"2024-02-10T08:22:35.807980+0000","last_unstale":"2024-02-10T08:24:00.795445+0000","last_undegraded":"2024-02-10T08:24:00.795445+0000","last_fullsized":"2024-02-10T08:24:00.795445+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":53,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.790237+0000","last_change":"2024-02-10T08:22:38.134900+0000","last_active":"2024-02-10T08:24:00.790237+0000","last_peered":"2024-02-10T08:24:00.790237+0000","last_clean":"2024-02-10T08:24:00.790237+0000","last_became_active":"2024-02-10T08:22:38.134256+0000","last_became_peered":"2024-02-10T08:22:38.134256+0000","last_unstale":"2024-02-10T08:24:00.790237+0000","last_undegraded":"2024-02-10T08:24:00.790237+0000","last_fullsized":"2024-02-10T08:24:00.790237+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.800113+0000","last_change":"2024-02-10T08:22:39.853556+0000","last_active":"2024-02-10T08:24:00.800113+0000","last_peered":"2024-02-10T08:24:00.800113+0000","last_clean":"2024-02-10T08:24:00.800113+0000","last_became_active":"2024-02-10T08:22:39.853204+0000","last_became_peered":"2024-02-10T08:22:39.853204+0000","last_unstale":"2024-02-10T08:24:00.800113+0000","last_undegraded":"2024-02-10T08:24:00.800113+0000","last_fullsized":"2024-02-10T08:24:00.800113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789179+0000","last_change":"2024-02-10T08:22:41.912577+0000","last_active":"2024-02-10T08:24:00.789179+0000","last_peered":"2024-02-10T08:24:00.789179+0000","last_clean":"2024-02-10T08:24:00.789179+0000","last_became_active":"2024-02-10T08:22:41.912282+0000","last_became_peered":"2024-02-10T08:22:41.912282+0000","last_unstale":"2024-02-10T08:24:00.789179+0000","last_undegraded":"2024-02-10T08:24:00.789179+0000","last_fullsized":"2024-02-10T08:24:00.789179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.802590+0000","last_change":"2024-02-10T08:22:35.811409+0000","last_active":"2024-02-10T08:24:00.802590+0000","last_peered":"2024-02-10T08:24:00.802590+0000","last_clean":"2024-02-10T08:24:00.802590+0000","last_became_active":"2024-02-10T08:22:35.810384+0000","last_became_peered":"2024-02-10T08:22:35.810384+0000","last_unstale":"2024-02-10T08:24:00.802590+0000","last_undegraded":"2024-02-10T08:24:00.802590+0000","last_fullsized":"2024-02-10T08:24:00.802590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795684+0000","last_change":"2024-02-10T08:22:37.833208+0000","last_active":"2024-02-10T08:24:00.795684+0000","last_peered":"2024-02-10T08:24:00.795684+0000","last_clean":"2024-02-10T08:24:00.795684+0000","last_became_active":"2024-02-10T08:22:37.832830+0000","last_became_peered":"2024-02-10T08:22:37.832830+0000","last_unstale":"2024-02-10T08:24:00.795684+0000","last_undegraded":"2024-02-10T08:24:00.795684+0000","last_fullsized":"2024-02-10T08:24:00.795684+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.535796+0000","last_change":"2024-02-10T08:22:39.857913+0000","last_active":"2024-02-10T08:27:09.535796+0000","last_peered":"2024-02-10T08:27:09.535796+0000","last_clean":"2024-02-10T08:27:09.535796+0000","last_became_active":"2024-02-10T08:22:39.857586+0000","last_became_peered":"2024-02-10T08:22:39.857586+0000","last_unstale":"2024-02-10T08:27:09.535796+0000","last_undegraded":"2024-02-10T08:27:09.535796+0000","last_fullsized":"2024-02-10T08:27:09.535796+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.789046+0000","last_change":"2024-02-10T08:22:41.908659+0000","last_active":"2024-02-10T08:24:00.789046+0000","last_peered":"2024-02-10T08:24:00.789046+0000","last_clean":"2024-02-10T08:24:00.789046+0000","last_became_active":"2024-02-10T08:22:41.908344+0000","last_became_peered":"2024-02-10T08:22:41.908344+0000","last_unstale":"2024-02-10T08:24:00.789046+0000","last_undegraded":"2024-02-10T08:24:00.789046+0000","last_fullsized":"2024-02-10T08:24:00.789046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.791111+0000","last_change":"2024-02-10T08:22:35.817072+0000","last_active":"2024-02-10T08:24:00.791111+0000","last_peered":"2024-02-10T08:24:00.791111+0000","last_clean":"2024-02-10T08:24:00.791111+0000","last_became_active":"2024-02-10T08:22:35.816735+0000","last_became_peered":"2024-02-10T08:22:35.816735+0000","last_unstale":"2024-02-10T08:24:00.791111+0000","last_undegraded":"2024-02-10T08:24:00.791111+0000","last_fullsized":"2024-02-10T08:24:00.791111+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796053+0000","last_change":"2024-02-10T08:22:37.817785+0000","last_active":"2024-02-10T08:24:00.796053+0000","last_peered":"2024-02-10T08:24:00.796053+0000","last_clean":"2024-02-10T08:24:00.796053+0000","last_became_active":"2024-02-10T08:22:37.817401+0000","last_became_peered":"2024-02-10T08:22:37.817401+0000","last_unstale":"2024-02-10T08:24:00.796053+0000","last_undegraded":"2024-02-10T08:24:00.796053+0000","last_fullsized":"2024-02-10T08:24:00.796053+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536449+0000","last_change":"2024-02-10T08:22:39.848618+0000","last_active":"2024-02-10T08:27:09.536449+0000","last_peered":"2024-02-10T08:27:09.536449+0000","last_clean":"2024-02-10T08:27:09.536449+0000","last_became_active":"2024-02-10T08:22:39.848241+0000","last_became_peered":"2024-02-10T08:22:39.848241+0000","last_unstale":"2024-02-10T08:27:09.536449+0000","last_undegraded":"2024-02-10T08:27:09.536449+0000","last_fullsized":"2024-02-10T08:27:09.536449+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796326+0000","last_change":"2024-02-10T08:22:41.910884+0000","last_active":"2024-02-10T08:24:00.796326+0000","last_peered":"2024-02-10T08:24:00.796326+0000","last_clean":"2024-02-10T08:24:00.796326+0000","last_became_active":"2024-02-10T08:22:41.910552+0000","last_became_peered":"2024-02-10T08:22:41.910552+0000","last_unstale":"2024-02-10T08:24:00.796326+0000","last_undegraded":"2024-02-10T08:24:00.796326+0000","last_fullsized":"2024-02-10T08:24:00.796326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.788356+0000","last_change":"2024-02-10T08:22:35.810186+0000","last_active":"2024-02-10T08:24:00.788356+0000","last_peered":"2024-02-10T08:24:00.788356+0000","last_clean":"2024-02-10T08:24:00.788356+0000","last_became_active":"2024-02-10T08:22:35.809503+0000","last_became_peered":"2024-02-10T08:22:35.809503+0000","last_unstale":"2024-02-10T08:24:00.788356+0000","last_undegraded":"2024-02-10T08:24:00.788356+0000","last_fullsized":"2024-02-10T08:24:00.788356+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787113+0000","last_change":"2024-02-10T08:22:38.135493+0000","last_active":"2024-02-10T08:24:00.787113+0000","last_peered":"2024-02-10T08:24:00.787113+0000","last_clean":"2024-02-10T08:24:00.787113+0000","last_became_active":"2024-02-10T08:22:38.135102+0000","last_became_peered":"2024-02-10T08:22:38.135102+0000","last_unstale":"2024-02-10T08:24:00.787113+0000","last_undegraded":"2024-02-10T08:24:00.787113+0000","last_fullsized":"2024-02-10T08:24:00.787113+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795961+0000","last_change":"2024-02-10T08:22:39.848407+0000","last_active":"2024-02-10T08:24:00.795961+0000","last_peered":"2024-02-10T08:24:00.795961+0000","last_clean":"2024-02-10T08:24:00.795961+0000","last_became_active":"2024-02-10T08:22:39.848078+0000","last_became_peered":"2024-02-10T08:22:39.848078+0000","last_unstale":"2024-02-10T08:24:00.795961+0000","last_undegraded":"2024-02-10T08:24:00.795961+0000","last_fullsized":"2024-02-10T08:24:00.795961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.801940+0000","last_change":"2024-02-10T08:22:41.917331+0000","last_active":"2024-02-10T08:24:00.801940+0000","last_peered":"2024-02-10T08:24:00.801940+0000","last_clean":"2024-02-10T08:24:00.801940+0000","last_became_active":"2024-02-10T08:22:41.916987+0000","last_became_peered":"2024-02-10T08:22:41.916987+0000","last_unstale":"2024-02-10T08:24:00.801940+0000","last_undegraded":"2024-02-10T08:24:00.801940+0000","last_fullsized":"2024-02-10T08:24:00.801940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":81,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536206+0000","last_change":"2024-02-10T08:22:39.853321+0000","last_active":"2024-02-10T08:27:09.536206+0000","last_peered":"2024-02-10T08:27:09.536206+0000","last_clean":"2024-02-10T08:27:09.536206+0000","last_became_active":"2024-02-10T08:22:39.852888+0000","last_became_peered":"2024-02-10T08:22:39.852888+0000","last_unstale":"2024-02-10T08:27:09.536206+0000","last_undegraded":"2024-02-10T08:27:09.536206+0000","last_fullsized":"2024-02-10T08:27:09.536206+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.786954+0000","last_change":"2024-02-10T08:22:35.801751+0000","last_active":"2024-02-10T08:24:00.786954+0000","last_peered":"2024-02-10T08:24:00.786954+0000","last_clean":"2024-02-10T08:24:00.786954+0000","last_became_active":"2024-02-10T08:22:35.801114+0000","last_became_peered":"2024-02-10T08:22:35.801114+0000","last_unstale":"2024-02-10T08:24:00.786954+0000","last_undegraded":"2024-02-10T08:24:00.786954+0000","last_fullsized":"2024-02-10T08:24:00.786954+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:24:03.002621+0000","last_change":"2024-02-10T08:24:03.002621+0000","last_active":"2024-02-10T08:24:03.002621+0000","last_peered":"2024-02-10T08:24:03.002621+0000","last_clean":"2024-02-10T08:24:03.002621+0000","last_became_active":"2024-02-10T08:24:01.796152+0000","last_became_peered":"2024-02-10T08:24:01.796152+0000","last_unstale":"2024-02-10T08:24:03.002621+0000","last_undegraded":"2024-02-10T08:24:03.002621+0000","last_fullsized":"2024-02-10T08:24:03.002621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787435+0000","last_change":"2024-02-10T08:22:41.915223+0000","last_active":"2024-02-10T08:24:00.787435+0000","last_peered":"2024-02-10T08:24:00.787435+0000","last_clean":"2024-02-10T08:24:00.787435+0000","last_became_active":"2024-02-10T08:22:41.914367+0000","last_became_peered":"2024-02-10T08:22:41.914367+0000","last_unstale":"2024-02-10T08:24:00.787435+0000","last_undegraded":"2024-02-10T08:24:00.787435+0000","last_fullsized":"2024-02-10T08:24:00.787435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.803426+0000","last_change":"2024-02-10T08:22:39.853353+0000","last_active":"2024-02-10T08:24:00.803426+0000","last_peered":"2024-02-10T08:24:00.803426+0000","last_clean":"2024-02-10T08:24:00.803426+0000","last_became_active":"2024-02-10T08:22:39.853038+0000","last_became_peered":"2024-02-10T08:22:39.853038+0000","last_unstale":"2024-02-10T08:24:00.803426+0000","last_undegraded":"2024-02-10T08:24:00.803426+0000","last_fullsized":"2024-02-10T08:24:00.803426+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796132+0000","last_change":"2024-02-10T08:22:35.810465+0000","last_active":"2024-02-10T08:24:00.796132+0000","last_peered":"2024-02-10T08:24:00.796132+0000","last_clean":"2024-02-10T08:24:00.796132+0000","last_became_active":"2024-02-10T08:22:35.809917+0000","last_became_peered":"2024-02-10T08:22:35.809917+0000","last_unstale":"2024-02-10T08:24:00.796132+0000","last_undegraded":"2024-02-10T08:24:00.796132+0000","last_fullsized":"2024-02-10T08:24:00.796132+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":36,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796599+0000","last_change":"2024-02-10T08:22:37.825011+0000","last_active":"2024-02-10T08:24:00.796599+0000","last_peered":"2024-02-10T08:24:00.796599+0000","last_clean":"2024-02-10T08:24:00.796599+0000","last_became_active":"2024-02-10T08:22:37.824643+0000","last_became_peered":"2024-02-10T08:22:37.824643+0000","last_unstale":"2024-02-10T08:24:00.796599+0000","last_undegraded":"2024-02-10T08:24:00.796599+0000","last_fullsized":"2024-02-10T08:24:00.796599+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.793346+0000","last_change":"2024-02-10T08:22:41.907067+0000","last_active":"2024-02-10T08:24:00.793346+0000","last_peered":"2024-02-10T08:24:00.793346+0000","last_clean":"2024-02-10T08:24:00.793346+0000","last_became_active":"2024-02-10T08:22:41.906714+0000","last_became_peered":"2024-02-10T08:22:41.906714+0000","last_unstale":"2024-02-10T08:24:00.793346+0000","last_undegraded":"2024-02-10T08:24:00.793346+0000","last_fullsized":"2024-02-10T08:24:00.793346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.795555+0000","last_change":"2024-02-10T08:22:39.847053+0000","last_active":"2024-02-10T08:24:00.795555+0000","last_peered":"2024-02-10T08:24:00.795555+0000","last_clean":"2024-02-10T08:24:00.795555+0000","last_became_active":"2024-02-10T08:22:39.846615+0000","last_became_peered":"2024-02-10T08:22:39.846615+0000","last_unstale":"2024-02-10T08:24:00.795555+0000","last_undegraded":"2024-02-10T08:24:00.795555+0000","last_fullsized":"2024-02-10T08:24:00.795555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.799717+0000","last_change":"2024-02-10T08:22:35.803370+0000","last_active":"2024-02-10T08:24:00.799717+0000","last_peered":"2024-02-10T08:24:00.799717+0000","last_clean":"2024-02-10T08:24:00.799717+0000","last_became_active":"2024-02-10T08:22:35.802904+0000","last_became_peered":"2024-02-10T08:22:35.802904+0000","last_unstale":"2024-02-10T08:24:00.799717+0000","last_undegraded":"2024-02-10T08:24:00.799717+0000","last_fullsized":"2024-02-10T08:24:00.799717+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:24:01.808888+0000","last_change":"2024-02-10T08:24:01.808888+0000","last_active":"2024-02-10T08:24:01.808888+0000","last_peered":"2024-02-10T08:24:01.808888+0000","last_clean":"2024-02-10T08:24:01.808888+0000","last_became_active":"2024-02-10T08:24:01.797575+0000","last_became_peered":"2024-02-10T08:24:01.797575+0000","last_unstale":"2024-02-10T08:24:01.808888+0000","last_undegraded":"2024-02-10T08:24:01.808888+0000","last_fullsized":"2024-02-10T08:24:01.808888+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.797034+0000","last_change":"2024-02-10T08:22:41.910486+0000","last_active":"2024-02-10T08:24:00.797034+0000","last_peered":"2024-02-10T08:24:00.797034+0000","last_clean":"2024-02-10T08:24:00.797034+0000","last_became_active":"2024-02-10T08:22:41.910065+0000","last_became_peered":"2024-02-10T08:22:41.910065+0000","last_unstale":"2024-02-10T08:24:00.797034+0000","last_undegraded":"2024-02-10T08:24:00.797034+0000","last_fullsized":"2024-02-10T08:24:00.797034+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:40.880085+0000","last_clean_scrub_stamp":"2024-02-10T08:22:40.880085+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":78,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-10T08:27:09.536045+0000","last_change":"2024-02-10T08:22:39.858880+0000","last_active":"2024-02-10T08:27:09.536045+0000","last_peered":"2024-02-10T08:27:09.536045+0000","last_clean":"2024-02-10T08:27:09.536045+0000","last_became_active":"2024-02-10T08:22:39.858514+0000","last_became_peered":"2024-02-10T08:22:39.858514+0000","last_unstale":"2024-02-10T08:27:09.536045+0000","last_undegraded":"2024-02-10T08:27:09.536045+0000","last_fullsized":"2024-02-10T08:27:09.536045+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:38.821421+0000","last_clean_scrub_stamp":"2024-02-10T08:22:38.821421+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.787589+0000","last_change":"2024-02-10T08:22:35.811552+0000","last_active":"2024-02-10T08:24:00.787589+0000","last_peered":"2024-02-10T08:24:00.787589+0000","last_clean":"2024-02-10T08:24:00.787589+0000","last_became_active":"2024-02-10T08:22:35.810643+0000","last_became_peered":"2024-02-10T08:22:35.810643+0000","last_unstale":"2024-02-10T08:24:00.787589+0000","last_undegraded":"2024-02-10T08:24:00.787589+0000","last_fullsized":"2024-02-10T08:24:00.787589+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:34.781629+0000","last_clean_scrub_stamp":"2024-02-10T08:22:34.781629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":32,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-02-10T08:24:00.796530+0000","last_change":"2024-02-10T08:22:37.822439+0000","last_active":"2024-02-10T08:24:00.796530+0000","last_peered":"2024-02-10T08:24:00.796530+0000","last_clean":"2024-02-10T08:24:00.796530+0000","last_became_active":"2024-02-10T08:22:37.822107+0000","last_became_peered":"2024-02-10T08:22:37.822107+0000","last_unstale":"2024-02-10T08:24:00.796530+0000","last_undegraded":"2024-02-10T08:24:00.796530+0000","last_fullsized":"2024-02-10T08:24:00.796530+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T08:22:36.798139+0000","last_clean_scrub_stamp":"2024-02-10T08:22:36.798139+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":272,"num_read_kb":267,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447368,"statfs":{"total":95995035648,"available":95690104832,"internally_reserved":0,"allocated":704512,"data_stored":289820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.86699999999999999,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.115,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90000000000000002,"15min":0.86899999999999999},"min":{"1min":0.69699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2310000000000001,"5min":1.319,"15min":1.319},"last":1.004}]},{"osd":1,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.86599999999999999,"15min":0.84399999999999997},"min":{"1min":0.70699999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.095},{"interface":"front","average":{"1min":0.996,"5min":0.92900000000000005,"15min":0.90400000000000003},"min":{"1min":0.83499999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.274,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.2969999999999999}]},{"osd":2,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.88500000000000001,"15min":0.87},"min":{"1min":0.69799999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1100000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.69199999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.351,"15min":1.351},"last":0.83899999999999997}]},{"osd":3,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89000000000000001,"15min":0.86099999999999999},"min":{"1min":0.71999999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.2430000000000001,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.96299999999999997,"15min":0.95199999999999996},"min":{"1min":0.79800000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.216,"5min":1.284,"15min":1.284},"last":0.97599999999999998}]},{"osd":4,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.84799999999999998,"15min":0.82499999999999996},"min":{"1min":0.65500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.212,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.87,"15min":0.84199999999999997},"min":{"1min":0.748,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1719999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.1379999999999999}]},{"osd":5,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.85699999999999998,"15min":0.81200000000000006},"min":{"1min":0.76400000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1919999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.034,"5min":0.93300000000000005,"15min":0.88300000000000001},"min":{"1min":0.74099999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.115}]},{"osd":6,"last update":"Sat Feb 10 08:26:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.99199999999999999,"15min":0.94399999999999995},"min":{"1min":0.57399999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3500000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":1.246},{"interface":"front","average":{"1min":1.0469999999999999,"5min":0.97699999999999998,"15min":0.94699999999999995},"min":{"1min":0.71799999999999997,"5min":0.501,"15min":0.501},"max":{"1min":1.3080000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.2250000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626502,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297752,"kb_used_data":656,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447400,"statfs":{"total":95995035648,"available":95690137600,"internally_reserved":0,"allocated":671744,"data_stored":287562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.95999999999999996,"15min":0.94499999999999995},"min":{"1min":0.68600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2769999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95799999999999996,"15min":0.95199999999999996},"min":{"1min":0.68000000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2689999999999999,"5min":1.95,"15min":1.95},"last":1.022}]},{"osd":1,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.90200000000000002,"15min":0.90000000000000002},"min":{"1min":0.73599999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.379,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.0089999999999999,"15min":1.002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.272,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.117}]},{"osd":2,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.101,"5min":1.135,"15min":1.135},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.88500000000000001},"min":{"1min":0.78000000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.246,"5min":1.28,"15min":1.28},"last":1}]},{"osd":3,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.042,"15min":1.036},"min":{"1min":0.88200000000000001,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3160000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.93799999999999994,"15min":0.93000000000000005},"min":{"1min":0.70499999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.1559999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86099999999999999,"15min":0.84499999999999997},"min":{"1min":0.64200000000000002,"5min":0.438,"15min":0.438},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.754},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98899999999999999,"15min":0.97199999999999998},"min":{"1min":0.70499999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.264,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2589999999999999}]},{"osd":5,"last update":"Sat Feb 10 08:26:43 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.878},"min":{"1min":0.66000000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.155,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0580000000000001,"15min":0.998},"min":{"1min":0.79900000000000004,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.242,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.3009999999999999}]},{"osd":7,"last update":"Sat Feb 10 08:26:25 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.997,"15min":0.94899999999999995},"min":{"1min":0.67800000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.4419999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1839999999999999,"15min":1.1599999999999999},"min":{"1min":0.92200000000000004,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.3440000000000001,"5min":1.605,"15min":1.605},"last":1.3420000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447348,"statfs":{"total":95995035648,"available":95690084352,"internally_reserved":0,"allocated":724992,"data_stored":289296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.748,"15min":0.61599999999999999},"min":{"1min":0.51900000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99299999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.65300000000000002},"min":{"1min":0.51600000000000001,"5min":0.42399999999999999,"15min":0.34599999999999997},"max":{"1min":0.96999999999999997,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Sat Feb 10 08:27:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80900000000000005,"15min":0.70099999999999996},"min":{"1min":0.41599999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1060000000000001,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79400000000000004,"15min":0.69499999999999995},"min":{"1min":0.57499999999999996,"5min":0.38,"15min":0.34799999999999998},"max":{"1min":0.94099999999999995,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.86799999999999999}]},{"osd":3,"last update":"Sat Feb 10 08:26:57 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.88200000000000001,"15min":0.82999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.079,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.216},{"interface":"front","average":{"1min":0.752,"5min":0.79700000000000004,"15min":0.77600000000000002},"min":{"1min":0.51600000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99099999999999999,"5min":1.621,"15min":1.621},"last":0.83699999999999997}]},{"osd":4,"last update":"Sat Feb 10 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97399999999999998,"15min":0.95399999999999996},"min":{"1min":0.68999999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3220000000000001,"5min":1.859,"15min":1.859},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0760000000000001,"15min":1.01},"min":{"1min":0.76400000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3460000000000001,"5min":1.488,"15min":1.488},"last":1.163}]},{"osd":5,"last update":"Sat Feb 10 08:27:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.998,"15min":0.998},"min":{"1min":0.72799999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5029999999999999,"5min":1.694,"15min":1.694},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.0840000000000001,"15min":1.0600000000000001},"min":{"1min":0.82599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.2090000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":1.0800000000000001}]},{"osd":6,"last update":"Sat Feb 10 08:26:47 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.139,"15min":1.1799999999999999},"min":{"1min":0.83999999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.079,"15min":1.1459999999999999},"min":{"1min":0.77800000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5369999999999999,"5min":2.109,"15min":2.109},"last":1.3560000000000001}]},{"osd":7,"last update":"Sat Feb 10 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.149,"15min":1.167},"min":{"1min":0.877,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.478,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.216,"15min":1.252},"min":{"1min":0.80000000000000004,"5min":0.627,"15min":0.627},"max":{"1min":1.4339999999999999,"5min":1.659,"15min":1.659},"last":1.4850000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297784,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447368,"statfs":{"total":95995035648,"available":95690104832,"internally_reserved":0,"allocated":704512,"data_stored":289517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.61799999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.34799999999999998},"max":{"1min":1.1100000000000001,"5min":1.609,"15min":1.609},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.64500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.92200000000000004,"5min":2.319,"15min":2.319},"last":0.85299999999999998}]},{"osd":2,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75800000000000001,"15min":0.66400000000000003},"min":{"1min":0.57999999999999996,"5min":0.497,"15min":0.40500000000000003},"max":{"1min":1.2,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83499999999999996,"15min":0.72599999999999998},"min":{"1min":0.58499999999999996,"5min":0.47399999999999998,"15min":0.34399999999999997},"max":{"1min":1.044,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Feb 10 08:26:58 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.871,"15min":0.79700000000000004},"min":{"1min":0.55100000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3500000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.873,"15min":0.83599999999999997},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.41499999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.92100000000000004}]},{"osd":4,"last update":"Sat Feb 10 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.026,"15min":0.98799999999999999},"min":{"1min":0.69499999999999995,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.3169999999999999,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0569999999999999,"15min":0.97699999999999998},"min":{"1min":0.68200000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3029999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.95899999999999996}]},{"osd":5,"last update":"Sat Feb 10 08:27:10 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.0760000000000001,"15min":1.038},"min":{"1min":0.84399999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5129999999999999,"5min":1.546,"15min":1.546},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.133,"15min":1.071},"min":{"1min":0.98199999999999998,"5min":0.69999999999999996,"15min":0.64200000000000002},"max":{"1min":1.571,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":1.1870000000000001}]},{"osd":6,"last update":"Sat Feb 10 08:26:47 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.1479999999999999,"15min":1.1459999999999999},"min":{"1min":0.90100000000000002,"5min":0.754,"15min":0.754},"max":{"1min":1.4990000000000001,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.1399999999999999,"15min":1.1539999999999999},"min":{"1min":0.88100000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.548,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.456}]},{"osd":7,"last update":"Sat Feb 10 08:26:26 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1990000000000001,"15min":1.2130000000000001},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.3109999999999999,"5min":1.885,"15min":1.885},"last":1.302},{"interface":"front","average":{"1min":1.198,"5min":1.226,"15min":1.266},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5820000000000001,"5min":2.4279999999999999,"15min":2.4279999999999999},"last":1.494}]}]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297768,"kb_used_data":672,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447384,"statfs":{"total":95995035648,"available":95690121216,"internally_reserved":0,"allocated":688128,"data_stored":288588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.64200000000000002},"min":{"1min":0.55100000000000005,"5min":0.38700000000000001,"15min":0.36599999999999999},"max":{"1min":1.046,"5min":1.391,"15min":1.391},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.80300000000000005,"15min":0.70399999999999996},"min":{"1min":0.47999999999999998,"5min":0.434,"15min":0.40899999999999997},"max":{"1min":0.95799999999999996,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.77000000000000002}]},{"osd":1,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78200000000000003,"15min":0.69399999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.143,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80300000000000005,"15min":0.73399999999999999},"min":{"1min":0.63,"5min":0.51600000000000001,"15min":0.48199999999999998},"max":{"1min":0.96799999999999997,"5min":1.661,"15min":1.661},"last":0.95799999999999996}]},{"osd":3,"last update":"Sat Feb 10 08:26:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.85299999999999998,"15min":0.85899999999999999},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.208,"5min":2.496,"15min":2.496},"last":1.05},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.86499999999999999,"15min":0.87},"min":{"1min":0.56599999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.141,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.0229999999999999}]},{"osd":4,"last update":"Sat Feb 10 08:26:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0309999999999999,"15min":1.0029999999999999},"min":{"1min":0.79500000000000004,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.526,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.028,"15min":0.97999999999999998},"min":{"1min":0.74299999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.395,"5min":1.504,"15min":1.504},"last":1.335}]},{"osd":5,"last update":"Sat Feb 10 08:27:04 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.075,"15min":1.0640000000000001},"min":{"1min":0.80600000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.3360000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.415},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.099,"15min":1.093},"min":{"1min":0.76100000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.514,"5min":1.603,"15min":1.698},"last":1.1020000000000001}]},{"osd":6,"last update":"Sat Feb 10 08:26:50 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.133,"15min":1.1140000000000001},"min":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.472,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2030000000000001,"15min":1.226},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.4350000000000001,"5min":1.708,"15min":1.708},"last":1.4870000000000001}]},{"osd":7,"last update":"Sat Feb 10 08:26:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.91800000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.514,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1870000000000001,"15min":1.1950000000000001},"min":{"1min":0.91000000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.611,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.3799999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297792,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447360,"statfs":{"total":95995035648,"available":95690096640,"internally_reserved":0,"allocated":712704,"data_stored":288741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.74199999999999999,"15min":0.68000000000000005},"min":{"1min":0.57499999999999996,"5min":0.47899999999999998,"15min":0.40899999999999997},"max":{"1min":1.042,"5min":1.262,"15min":1.262},"last":0.628},{"interface":"front","average":{"1min":0.75,"5min":0.75800000000000001,"15min":0.753},"min":{"1min":0.58599999999999997,"5min":0.249,"15min":0.249},"max":{"1min":1.0800000000000001,"5min":1.524,"15min":1.524},"last":0.873}]},{"osd":1,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82699999999999996,"15min":0.76500000000000001},"min":{"1min":0.66900000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0780000000000001,"5min":1.617,"15min":1.617},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80900000000000005,"15min":0.753},"min":{"1min":0.56299999999999994,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.83599999999999997}]},{"osd":2,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.91600000000000004,"15min":0.84899999999999998},"min":{"1min":0.69299999999999995,"5min":0.55900000000000005,"15min":0.49399999999999999},"max":{"1min":1.236,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91700000000000004,"15min":0.86599999999999999},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.216,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.111}]},{"osd":4,"last update":"Sat Feb 10 08:26:21 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0029999999999999,"15min":0.96299999999999997},"min":{"1min":0.77700000000000002,"5min":0.67500000000000004,"15min":0.67200000000000004},"max":{"1min":1.51,"5min":1.657,"15min":1.657},"last":1.22},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.046,"15min":1.01},"min":{"1min":0.78600000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.97999999999999998}]},{"osd":5,"last update":"Sat Feb 10 08:27:11 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.73399999999999999,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3680000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.139,"5min":1.0940000000000001,"15min":1.0760000000000001},"min":{"1min":0.91300000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.4259999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0349999999999999}]},{"osd":6,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1279999999999999,"15min":1.1240000000000001},"min":{"1min":0.94699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1200000000000001,"15min":1.0940000000000001},"min":{"1min":0.85199999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.468,"5min":1.675,"15min":1.675},"last":1.294}]},{"osd":7,"last update":"Sat Feb 10 08:26:34 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.93500000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.1699999999999999,"15min":1.1259999999999999},"min":{"1min":1.0289999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.8200000000000001,"5min":1.8200000000000001,"15min":1.8200000000000001},"last":1.1679999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018965,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297788,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447364,"statfs":{"total":95995035648,"available":95690100736,"internally_reserved":0,"allocated":708608,"data_stored":289295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.89000000000000001,"15min":0.88},"min":{"1min":0.53900000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.26,"5min":1.921,"15min":1.921},"last":1.073},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.879,"15min":0.84599999999999997},"min":{"1min":0.73199999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.26,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.159}]},{"osd":1,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.89200000000000002,"15min":0.85999999999999999},"min":{"1min":0.69499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.1839999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.93400000000000005,"15min":0.88900000000000001},"min":{"1min":0.74299999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1120000000000001,"5min":1.409,"15min":1.409},"last":0.83199999999999996}]},{"osd":2,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.621,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.1799999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.012},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93700000000000006,"15min":0.90600000000000003},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.206,"5min":1.397,"15min":1.397},"last":1.101}]},{"osd":3,"last update":"Sat Feb 10 08:26:19 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.96499999999999997,"15min":0.91500000000000004},"min":{"1min":0.84699999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.1839999999999999,"5min":1.968,"15min":1.968},"last":1.2},{"interface":"front","average":{"1min":1.056,"5min":0.95699999999999996,"15min":0.94199999999999995},"min":{"1min":0.91300000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2050000000000001,"5min":1.337,"15min":1.337},"last":1.0529999999999999}]},{"osd":5,"last update":"Sat Feb 10 08:27:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82599999999999996,"15min":0.77200000000000002},"min":{"1min":0.64800000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.129,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1,"5min":0.96399999999999997,"15min":0.86299999999999999},"min":{"1min":0.70999999999999996,"5min":0.49399999999999999,"15min":0.433},"max":{"1min":1.3149999999999999,"5min":1.528,"15min":1.528},"last":0.88900000000000001}]},{"osd":6,"last update":"Sat Feb 10 08:26:46 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.94499999999999995,"15min":0.86899999999999999},"min":{"1min":0.73799999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.208,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.999,"15min":0.95099999999999996},"min":{"1min":0.67800000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.343,"5min":1.839,"15min":1.839},"last":0.92500000000000004}]},{"osd":7,"last update":"Sat Feb 10 08:26:25 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0800000000000001,"15min":1.0469999999999999},"min":{"1min":0.95299999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3520000000000001,"5min":2.117,"15min":2.117},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0509999999999999,"15min":1.0249999999999999},"min":{"1min":0.74299999999999999,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2689999999999999,"5min":1.698,"15min":1.698},"last":1.119}]}]},{"osd":5,"up_from":36,"seq":154618822735,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297764,"kb_used_data":668,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447388,"statfs":{"total":95995035648,"available":95690125312,"internally_reserved":0,"allocated":684032,"data_stored":288053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.92200000000000004,"15min":0.88},"min":{"1min":0.63100000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.238,"5min":1.282,"15min":1.282},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.64200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":1.0129999999999999}]},{"osd":1,"last update":"Sat Feb 10 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.90900000000000003,"15min":0.85899999999999999},"min":{"1min":0.67400000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.88300000000000001,"15min":0.86699999999999999},"min":{"1min":0.755,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1879999999999999,"5min":1.361,"15min":1.361},"last":1.1319999999999999}]},{"osd":2,"last update":"Sat Feb 10 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.91300000000000003,"15min":0.85199999999999998},"min":{"1min":0.64500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2110000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95699999999999996,"15min":0.94099999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.86899999999999999}]},{"osd":3,"last update":"Sat Feb 10 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.94099999999999995,"15min":0.85299999999999998},"min":{"1min":0.71499999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.02,"15min":0.96399999999999997},"min":{"1min":0.79200000000000004,"5min":0.67500000000000004,"15min":0.66100000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.0700000000000001}]},{"osd":4,"last update":"Sat Feb 10 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.85499999999999998,"15min":0.74199999999999999},"min":{"1min":0.68300000000000005,"5min":0.504,"15min":0.38900000000000001},"max":{"1min":1.2849999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":0.92700000000000005,"15min":0.76500000000000001},"min":{"1min":0.79700000000000004,"5min":0.48499999999999999,"15min":0.40500000000000003},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.92800000000000005}]},{"osd":6,"last update":"Sat Feb 10 08:26:54 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.94199999999999995,"15min":0.85699999999999998},"min":{"1min":0.70599999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.161},{"interface":"front","average":{"1min":1.121,"5min":0.96099999999999997,"15min":0.875},"min":{"1min":0.80800000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.371,"5min":1.375,"15min":1.375},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Feb 10 08:26:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":0.96499999999999997,"15min":0.88300000000000001},"min":{"1min":0.88800000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.032,"15min":0.94199999999999995},"min":{"1min":0.74199999999999999,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.96399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T08:27:21.965 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-10T08:27:21.965 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-10T08:27:21.965 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-10T08:27:21.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0e714d9a4bd2a821113e6318adb87bd06cf81ec1 shell --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 -- ceph health --format=json 2024-02-10T08:27:22.016 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:21 smithi067 bash[17385]: audit 2024-02-10T08:27:21.187377+0000 mgr.y (mgr.14547) 185 : audit [DBG] from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:27:22.017 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:21 smithi067 bash[24488]: audit 2024-02-10T08:27:21.187377+0000 mgr.y (mgr.14547) 185 : audit [DBG] from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:27:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:21 smithi134 bash[18722]: audit 2024-02-10T08:27:21.187377+0000 mgr.y (mgr.14547) 185 : audit [DBG] from='client.14751 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T08:27:22.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:27:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:27:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:23 smithi134 bash[18722]: audit 2024-02-10T08:27:21.787331+0000 mgr.y (mgr.14547) 186 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:23 smithi134 bash[18722]: cluster 2024-02-10T08:27:22.653217+0000 mgr.y (mgr.14547) 187 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:27:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:27:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:23 smithi067 bash[17385]: audit 2024-02-10T08:27:21.787331+0000 mgr.y (mgr.14547) 186 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:23 smithi067 bash[17385]: cluster 2024-02-10T08:27:22.653217+0000 mgr.y (mgr.14547) 187 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:23 smithi067 bash[24488]: audit 2024-02-10T08:27:21.787331+0000 mgr.y (mgr.14547) 186 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:23 smithi067 bash[24488]: cluster 2024-02-10T08:27:22.653217+0000 mgr.y (mgr.14547) 187 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:25.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:24 smithi134 bash[18722]: cluster 2024-02-10T08:27:24.654047+0000 mgr.y (mgr.14547) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:24 smithi067 bash[17385]: cluster 2024-02-10T08:27:24.654047+0000 mgr.y (mgr.14547) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:25.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:24 smithi067 bash[24488]: cluster 2024-02-10T08:27:24.654047+0000 mgr.y (mgr.14547) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T08:27:27.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:26 smithi134 bash[18722]: cluster 2024-02-10T08:27:26.655520+0000 mgr.y (mgr.14547) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:26 smithi067 bash[17385]: cluster 2024-02-10T08:27:26.655520+0000 mgr.y (mgr.14547) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:27.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:26 smithi067 bash[24488]: cluster 2024-02-10T08:27:26.655520+0000 mgr.y (mgr.14547) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:28.055 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:27:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:27] "GET /metrics HTTP/1.1" 200 172200 "" "Prometheus/2.33.4" 2024-02-10T08:27:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:28 smithi067 bash[17385]: cluster 2024-02-10T08:27:28.656196+0000 mgr.y (mgr.14547) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:28.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:28 smithi067 bash[24488]: cluster 2024-02-10T08:27:28.656196+0000 mgr.y (mgr.14547) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:28 smithi134 bash[18722]: cluster 2024-02-10T08:27:28.656196+0000 mgr.y (mgr.14547) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 -- 172.21.15.67:0/3129497218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 msgr2=0x7f4fc4104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 --2- 172.21.15.67:0/3129497218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc4104b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f4fbc009be0 tx=0x7f4fbc004d10).stop 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 -- 172.21.15.67:0/3129497218 shutdown_connections 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 --2- 172.21.15.67:0/3129497218 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc4104b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 --2- 172.21.15.67:0/3129497218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc40fdc90 0x7f4fc40fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 --2- 172.21.15.67:0/3129497218 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f4fc40fe5f0 0x7f4fc40f7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.637 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 -- 172.21.15.67:0/3129497218 >> 172.21.15.67:0/3129497218 conn(0x7f4fc40f38c0 msgr2=0x7f4fc40f5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:29.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 -- 172.21.15.67:0/3129497218 shutdown_connections 2024-02-10T08:27:29.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 -- 172.21.15.67:0/3129497218 wait complete. 2024-02-10T08:27:29.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.634+0000 7f4fc9959700 1 Processor -- start 2024-02-10T08:27:29.638 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- start start 2024-02-10T08:27:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f4fc40fdc90 0x7f4fc410e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:29.639 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc40fe5f0 0x7f4fc4107f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4108440 con 0x7f4fc40f8500 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc41085b0 con 0x7f4fc40fe5f0 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4108770 con 0x7f4fc40fdc90 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc410ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:47810/0 (socket says 172.21.15.67:47810) 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 -- 172.21.15.67:0/866921106 learned_addr learned my addr 172.21.15.67:0/866921106 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc27fc700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f4fc40fdc90 0x7f4fc410e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc37fe700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc40fe5f0 0x7f4fc4107f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:29.641 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 -- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f4fc40fdc90 msgr2=0x7f4fc410e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f4fc40fdc90 0x7f4fc410e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 -- 172.21.15.67:0/866921106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc40fe5f0 msgr2=0x7f4fc4107f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc40fe5f0 0x7f4fc4107f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 -- 172.21.15.67:0/866921106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0097c0 con 0x7f4fc40f8500 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc2ffd700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc410ddb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f4fac00e970 tx=0x7f4fac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00ee60 con 0x7f4fc40f8500 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc4108a50 con 0x7f4fc40f8500 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f4fac00f040 con 0x7f4fc40f8500 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc4108dd0 con 0x7f4fc40f8500 2024-02-10T08:27:29.642 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00c5b0 con 0x7f4fc40f8500 2024-02-10T08:27:29.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.638+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc4060740 con 0x7f4fc40f8500 2024-02-10T08:27:29.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.642+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f4fac007b40 con 0x7f4fc40f8500 2024-02-10T08:27:29.643 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.642+0000 7f4fbbfff700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f4fb0061340 0x7f4fb0063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:27:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.642+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7392+0+0 (secure 0 0 0) 0x7f4fac07cc50 con 0x7f4fc40f8500 2024-02-10T08:27:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.642+0000 7f4fc27fc700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f4fb0061340 0x7f4fb0063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:27:29.644 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.642+0000 7f4fc27fc700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f4fb0061340 0x7f4fb0063800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f4fc41096f0 tx=0x7f4fb400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:27:29.650 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.646+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f4fac04af60 con 0x7f4fc40f8500 2024-02-10T08:27:29.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.874+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4fc40475e0 con 0x7f4fc40f8500 2024-02-10T08:27:29.878 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.874+0000 7f4fbbfff700 1 -- 172.21.15.67:0/866921106 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f4fac019040 con 0x7f4fc40f8500 2024-02-10T08:27:29.878 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f4fb0061340 msgr2=0x7f4fb0063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f4fb0061340 0x7f4fb0063800 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f4fc41096f0 tx=0x7f4fb400b040).stop 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 msgr2=0x7f4fc410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc410ddb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f4fac00e970 tx=0x7f4fac00b040).stop 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 shutdown_connections 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f4fc40f8500 0x7f4fc410ddb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f4fb0061340 0x7f4fb0063800 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fc40fe5f0 0x7f4fc4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 --2- 172.21.15.67:0/866921106 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f4fc40fdc90 0x7f4fc410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 >> 172.21.15.67:0/866921106 conn(0x7f4fc40f38c0 msgr2=0x7f4fc40f5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:27:29.882 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 shutdown_connections 2024-02-10T08:27:29.883 INFO:teuthology.orchestra.run.smithi067.stderr:2024-02-10T08:27:29.878+0000 7f4fc9959700 1 -- 172.21.15.67:0/866921106 wait complete. 2024-02-10T08:27:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:29 smithi134 bash[18722]: audit 2024-02-10T08:27:29.878137+0000 mon.a (mon.0) 686 : audit [DBG] from='client.? 172.21.15.67:0/866921106' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T08:27:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:29 smithi067 bash[17385]: audit 2024-02-10T08:27:29.878137+0000 mon.a (mon.0) 686 : audit [DBG] from='client.? 172.21.15.67:0/866921106' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T08:27:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:29 smithi067 bash[24488]: audit 2024-02-10T08:27:29.878137+0000 mon.a (mon.0) 686 : audit [DBG] from='client.? 172.21.15.67:0/866921106' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T08:27:30.666 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-10T08:27:30.667 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-10T08:27:30.667 INFO:tasks.cephadm:Setup complete, yielding 2024-02-10T08:27:30.667 INFO:teuthology.run_tasks:Running task workunit... 2024-02-10T08:27:30.681 INFO:tasks.workunit:Pulling workunits from ref 57bd6abdec7bb457ae7999d9c96682e9ac678e27 2024-02-10T08:27:30.681 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-10T08:27:30.682 DEBUG:teuthology.orchestra.run.smithi067:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-10T08:27:30.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:27:30.718 INFO:teuthology.orchestra.run.smithi067.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-10T08:27:30.718 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-10T08:27:30.765 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-10T08:27:30.766 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-10T08:27:30.813 INFO:tasks.workunit:timeout=3h 2024-02-10T08:27:30.813 INFO:tasks.workunit:cleanup=True 2024-02-10T08:27:30.814 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ljflores/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 57bd6abdec7bb457ae7999d9c96682e9ac678e27 2024-02-10T08:27:30.863 INFO:tasks.workunit.client.0.smithi067.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-10T08:27:31.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:30 smithi134 bash[18722]: cluster 2024-02-10T08:27:30.656993+0000 mgr.y (mgr.14547) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:30 smithi067 bash[17385]: cluster 2024-02-10T08:27:30.656993+0000 mgr.y (mgr.14547) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:31.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:30 smithi067 bash[24488]: cluster 2024-02-10T08:27:30.656993+0000 mgr.y (mgr.14547) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:32.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:27:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:27:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:33 smithi134 bash[18722]: audit 2024-02-10T08:27:31.798893+0000 mgr.y (mgr.14547) 192 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:33 smithi134 bash[18722]: cluster 2024-02-10T08:27:32.658531+0000 mgr.y (mgr.14547) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:27:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:27:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:33 smithi067 bash[17385]: audit 2024-02-10T08:27:31.798893+0000 mgr.y (mgr.14547) 192 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:33 smithi067 bash[17385]: cluster 2024-02-10T08:27:32.658531+0000 mgr.y (mgr.14547) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:33.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:33 smithi067 bash[24488]: audit 2024-02-10T08:27:31.798893+0000 mgr.y (mgr.14547) 192 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:33.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:33 smithi067 bash[24488]: cluster 2024-02-10T08:27:32.658531+0000 mgr.y (mgr.14547) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:34 smithi134 bash[18722]: cluster 2024-02-10T08:27:34.659257+0000 mgr.y (mgr.14547) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:34 smithi067 bash[17385]: cluster 2024-02-10T08:27:34.659257+0000 mgr.y (mgr.14547) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:34 smithi067 bash[24488]: cluster 2024-02-10T08:27:34.659257+0000 mgr.y (mgr.14547) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:36 smithi134 bash[18722]: cluster 2024-02-10T08:27:36.661080+0000 mgr.y (mgr.14547) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:36 smithi067 bash[17385]: cluster 2024-02-10T08:27:36.661080+0000 mgr.y (mgr.14547) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:37.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:36 smithi067 bash[24488]: cluster 2024-02-10T08:27:36.661080+0000 mgr.y (mgr.14547) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:27:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:37] "GET /metrics HTTP/1.1" 200 172192 "" "Prometheus/2.33.4" 2024-02-10T08:27:38.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:38 smithi067 bash[17385]: audit 2024-02-10T08:27:38.269594+0000 mon.c (mon.2) 83 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:27:38.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:38 smithi067 bash[24488]: audit 2024-02-10T08:27:38.269594+0000 mon.c (mon.2) 83 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:27:38.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:38 smithi134 bash[18722]: audit 2024-02-10T08:27:38.269594+0000 mon.c (mon.2) 83 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[24488]: audit 2024-02-10T08:27:38.579530+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[24488]: audit 2024-02-10T08:27:38.580569+0000 mon.c (mon.2) 85 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[24488]: audit 2024-02-10T08:27:38.588001+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[24488]: cluster 2024-02-10T08:27:38.661631+0000 mgr.y (mgr.14547) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[17385]: audit 2024-02-10T08:27:38.579530+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[17385]: audit 2024-02-10T08:27:38.580569+0000 mon.c (mon.2) 85 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:27:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[17385]: audit 2024-02-10T08:27:38.588001+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:27:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:39 smithi067 bash[17385]: cluster 2024-02-10T08:27:38.661631+0000 mgr.y (mgr.14547) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:39 smithi134 bash[18722]: audit 2024-02-10T08:27:38.579530+0000 mon.c (mon.2) 84 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:27:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:39 smithi134 bash[18722]: audit 2024-02-10T08:27:38.580569+0000 mon.c (mon.2) 85 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:27:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:39 smithi134 bash[18722]: audit 2024-02-10T08:27:38.588001+0000 mon.a (mon.0) 687 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:27:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:39 smithi134 bash[18722]: cluster 2024-02-10T08:27:38.661631+0000 mgr.y (mgr.14547) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:40 smithi134 bash[18722]: cluster 2024-02-10T08:27:40.662230+0000 mgr.y (mgr.14547) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:40 smithi067 bash[17385]: cluster 2024-02-10T08:27:40.662230+0000 mgr.y (mgr.14547) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:40 smithi067 bash[24488]: cluster 2024-02-10T08:27:40.662230+0000 mgr.y (mgr.14547) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:42.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:27:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:27:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:43 smithi134 bash[18722]: audit 2024-02-10T08:27:41.810220+0000 mgr.y (mgr.14547) 198 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:43 smithi134 bash[18722]: cluster 2024-02-10T08:27:42.663479+0000 mgr.y (mgr.14547) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:43.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:27:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:27:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:43 smithi067 bash[17385]: audit 2024-02-10T08:27:41.810220+0000 mgr.y (mgr.14547) 198 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:43 smithi067 bash[17385]: cluster 2024-02-10T08:27:42.663479+0000 mgr.y (mgr.14547) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:43 smithi067 bash[24488]: audit 2024-02-10T08:27:41.810220+0000 mgr.y (mgr.14547) 198 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:43 smithi067 bash[24488]: cluster 2024-02-10T08:27:42.663479+0000 mgr.y (mgr.14547) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:45.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:44 smithi134 bash[18722]: cluster 2024-02-10T08:27:44.664096+0000 mgr.y (mgr.14547) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:44 smithi067 bash[17385]: cluster 2024-02-10T08:27:44.664096+0000 mgr.y (mgr.14547) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:45.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:44 smithi067 bash[24488]: cluster 2024-02-10T08:27:44.664096+0000 mgr.y (mgr.14547) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:46 smithi134 bash[18722]: cluster 2024-02-10T08:27:46.665422+0000 mgr.y (mgr.14547) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:46 smithi067 bash[17385]: cluster 2024-02-10T08:27:46.665422+0000 mgr.y (mgr.14547) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:47.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:46 smithi067 bash[24488]: cluster 2024-02-10T08:27:46.665422+0000 mgr.y (mgr.14547) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:27:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:47] "GET /metrics HTTP/1.1" 200 172195 "" "Prometheus/2.33.4" 2024-02-10T08:27:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:48 smithi134 bash[18722]: cluster 2024-02-10T08:27:48.666006+0000 mgr.y (mgr.14547) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:48 smithi067 bash[17385]: cluster 2024-02-10T08:27:48.666006+0000 mgr.y (mgr.14547) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:49.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:48 smithi067 bash[24488]: cluster 2024-02-10T08:27:48.666006+0000 mgr.y (mgr.14547) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:50 smithi134 bash[18722]: cluster 2024-02-10T08:27:50.666683+0000 mgr.y (mgr.14547) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:51.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:50 smithi067 bash[17385]: cluster 2024-02-10T08:27:50.666683+0000 mgr.y (mgr.14547) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:51.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:50 smithi067 bash[24488]: cluster 2024-02-10T08:27:50.666683+0000 mgr.y (mgr.14547) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:52.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:27:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:27:53.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:27:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:27:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:53 smithi134 bash[18722]: audit 2024-02-10T08:27:51.814238+0000 mgr.y (mgr.14547) 204 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:53 smithi134 bash[18722]: cluster 2024-02-10T08:27:52.667549+0000 mgr.y (mgr.14547) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:53 smithi067 bash[17385]: audit 2024-02-10T08:27:51.814238+0000 mgr.y (mgr.14547) 204 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:53 smithi067 bash[17385]: cluster 2024-02-10T08:27:52.667549+0000 mgr.y (mgr.14547) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:53 smithi067 bash[24488]: audit 2024-02-10T08:27:51.814238+0000 mgr.y (mgr.14547) 204 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:27:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:53 smithi067 bash[24488]: cluster 2024-02-10T08:27:52.667549+0000 mgr.y (mgr.14547) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:55.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:54 smithi134 bash[18722]: cluster 2024-02-10T08:27:54.667959+0000 mgr.y (mgr.14547) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:55.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:54 smithi067 bash[17385]: cluster 2024-02-10T08:27:54.667959+0000 mgr.y (mgr.14547) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:55.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:54 smithi067 bash[24488]: cluster 2024-02-10T08:27:54.667959+0000 mgr.y (mgr.14547) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:27:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:56 smithi134 bash[18722]: cluster 2024-02-10T08:27:56.668756+0000 mgr.y (mgr.14547) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:57.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:56 smithi067 bash[24488]: cluster 2024-02-10T08:27:56.668756+0000 mgr.y (mgr.14547) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:57.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:56 smithi067 bash[17385]: cluster 2024-02-10T08:27:56.668756+0000 mgr.y (mgr.14547) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:27:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:27:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:27:57] "GET /metrics HTTP/1.1" 200 172195 "" "Prometheus/2.33.4" 2024-02-10T08:27:59.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:27:58 smithi134 bash[18722]: cluster 2024-02-10T08:27:58.669341+0000 mgr.y (mgr.14547) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:27:59.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:27:58 smithi067 bash[17385]: cluster 2024-02-10T08:27:58.669341+0000 mgr.y (mgr.14547) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:27:59.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:27:58 smithi067 bash[24488]: cluster 2024-02-10T08:27:58.669341+0000 mgr.y (mgr.14547) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:01.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:00 smithi134 bash[18722]: cluster 2024-02-10T08:28:00.670101+0000 mgr.y (mgr.14547) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:00 smithi067 bash[17385]: cluster 2024-02-10T08:28:00.670101+0000 mgr.y (mgr.14547) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:01.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:00 smithi067 bash[24488]: cluster 2024-02-10T08:28:00.670101+0000 mgr.y (mgr.14547) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:02.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:28:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:28:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:03 smithi134 bash[18722]: audit 2024-02-10T08:28:01.823263+0000 mgr.y (mgr.14547) 210 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:03 smithi134 bash[18722]: cluster 2024-02-10T08:28:02.670956+0000 mgr.y (mgr.14547) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:03.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:28:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:28:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:03 smithi067 bash[17385]: audit 2024-02-10T08:28:01.823263+0000 mgr.y (mgr.14547) 210 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:03 smithi067 bash[17385]: cluster 2024-02-10T08:28:02.670956+0000 mgr.y (mgr.14547) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:03 smithi067 bash[24488]: audit 2024-02-10T08:28:01.823263+0000 mgr.y (mgr.14547) 210 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:03 smithi067 bash[24488]: cluster 2024-02-10T08:28:02.670956+0000 mgr.y (mgr.14547) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:05.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:04 smithi134 bash[18722]: cluster 2024-02-10T08:28:04.671366+0000 mgr.y (mgr.14547) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:05.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:04 smithi067 bash[24488]: cluster 2024-02-10T08:28:04.671366+0000 mgr.y (mgr.14547) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:04 smithi067 bash[17385]: cluster 2024-02-10T08:28:04.671366+0000 mgr.y (mgr.14547) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:06 smithi134 bash[18722]: cluster 2024-02-10T08:28:06.672602+0000 mgr.y (mgr.14547) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:07.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:06 smithi067 bash[17385]: cluster 2024-02-10T08:28:06.672602+0000 mgr.y (mgr.14547) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:07.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:06 smithi067 bash[24488]: cluster 2024-02-10T08:28:06.672602+0000 mgr.y (mgr.14547) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:28:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:07] "GET /metrics HTTP/1.1" 200 172198 "" "Prometheus/2.33.4" 2024-02-10T08:28:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:08 smithi134 bash[18722]: cluster 2024-02-10T08:28:08.673230+0000 mgr.y (mgr.14547) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:28:09.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:08 smithi067 bash[17385]: cluster 2024-02-10T08:28:08.673230+0000 mgr.y (mgr.14547) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:28:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:08 smithi067 bash[24488]: cluster 2024-02-10T08:28:08.673230+0000 mgr.y (mgr.14547) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:28:11.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:10 smithi134 bash[18722]: cluster 2024-02-10T08:28:10.673826+0000 mgr.y (mgr.14547) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:11.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:10 smithi067 bash[17385]: cluster 2024-02-10T08:28:10.673826+0000 mgr.y (mgr.14547) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:10 smithi067 bash[24488]: cluster 2024-02-10T08:28:10.673826+0000 mgr.y (mgr.14547) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-10T08:28:12.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:28:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:28:13.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:28:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:28:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:13 smithi134 bash[18722]: audit 2024-02-10T08:28:11.831044+0000 mgr.y (mgr.14547) 216 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:13 smithi134 bash[18722]: cluster 2024-02-10T08:28:12.675211+0000 mgr.y (mgr.14547) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:13 smithi067 bash[17385]: audit 2024-02-10T08:28:11.831044+0000 mgr.y (mgr.14547) 216 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:13 smithi067 bash[17385]: cluster 2024-02-10T08:28:12.675211+0000 mgr.y (mgr.14547) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:13 smithi067 bash[24488]: audit 2024-02-10T08:28:11.831044+0000 mgr.y (mgr.14547) 216 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:13 smithi067 bash[24488]: cluster 2024-02-10T08:28:12.675211+0000 mgr.y (mgr.14547) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:14 smithi134 bash[18722]: cluster 2024-02-10T08:28:14.675772+0000 mgr.y (mgr.14547) 218 : cluster [DBG] pgmap v162: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:28:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:14 smithi067 bash[17385]: cluster 2024-02-10T08:28:14.675772+0000 mgr.y (mgr.14547) 218 : cluster [DBG] pgmap v162: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:28:15.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:14 smithi067 bash[24488]: cluster 2024-02-10T08:28:14.675772+0000 mgr.y (mgr.14547) 218 : cluster [DBG] pgmap v162: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr:Note: switching to '57bd6abdec7bb457ae7999d9c96682e9ac678e27'. 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr:state without impacting any branches by switching back to a branch. 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-10T08:28:15.649 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr: git switch -c 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr:Or undo this operation with: 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr: git switch - 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr: 2024-02-10T08:28:15.650 INFO:tasks.workunit.client.0.smithi067.stderr:HEAD is now at 57bd6abdec7 qa/suites/orch: whitelist warnings that are expected in test environments 2024-02-10T08:28:15.656 DEBUG:teuthology.orchestra.run.smithi067:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-10T08:28:15.717 INFO:tasks.workunit.client.0.smithi067.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-10T08:28:15.720 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-10T08:28:15.720 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-10T08:28:16.416 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-10T08:28:16.509 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-10T08:28:16.574 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-10T08:28:16.576 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-10T08:28:16.576 INFO:tasks.workunit.client.0.smithi067.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-10T08:28:16.633 INFO:tasks.workunit.client.0.smithi067.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-10T08:28:16.637 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-10T08:28:16.637 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-10T08:28:16.685 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-10T08:28:16.686 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-10T08:28:16.687 DEBUG:teuthology.orchestra.run.smithi067:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57bd6abdec7bb457ae7999d9c96682e9ac678e27 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ parallel=1 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' '' = --serial ']' 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ color= 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -t 1 ']' 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ declare -A pids 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.740 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.741 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_aio 2024-02-10T08:28:16.741 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_aio' 2024-02-10T08:28:16.741 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_aio 2024-02-10T08:28:16.741 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.744 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_aio 2024-02-10T08:28:16.744 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54232 2024-02-10T08:28:16.744 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_aio on pid 54232' 2024-02-10T08:28:16.744 INFO:tasks.workunit.client.0.smithi067.stdout:test api_aio on pid 54232 2024-02-10T08:28:16.745 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54232 2024-02-10T08:28:16.745 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.745 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.745 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-10T08:28:16.745 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_aio_pp 2024-02-10T08:28:16.745 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_aio_pp' 2024-02-10T08:28:16.746 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_aio_pp 2024-02-10T08:28:16.746 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.746 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.747 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.747 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.747 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.747 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_aio 2024-02-10T08:28:16.747 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-10T08:28:16.747 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_aio: /' 2024-02-10T08:28:16.748 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_aio_pp 2024-02-10T08:28:16.749 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54240 2024-02-10T08:28:16.749 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_aio_pp on pid 54240' 2024-02-10T08:28:16.749 INFO:tasks.workunit.client.0.smithi067.stdout:test api_aio_pp on pid 54240 2024-02-10T08:28:16.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54240 2024-02-10T08:28:16.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-10T08:28:16.750 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_io 2024-02-10T08:28:16.750 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_io' 2024-02-10T08:28:16.751 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.751 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.751 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_io 2024-02-10T08:28:16.751 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.751 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.751 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.752 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_aio_pp 2024-02-10T08:28:16.752 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-10T08:28:16.752 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-10T08:28:16.754 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_io 2024-02-10T08:28:16.754 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54248 2024-02-10T08:28:16.754 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_io on pid 54248' 2024-02-10T08:28:16.754 INFO:tasks.workunit.client.0.smithi067.stdout:test api_io on pid 54248 2024-02-10T08:28:16.755 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54248 2024-02-10T08:28:16.755 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.755 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.755 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-10T08:28:16.755 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_io_pp 2024-02-10T08:28:16.756 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_io_pp' 2024-02-10T08:28:16.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.757 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.757 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_io_pp 2024-02-10T08:28:16.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_io 2024-02-10T08:28:16.758 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_io: /' 2024-02-10T08:28:16.758 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_io.log 2024-02-10T08:28:16.759 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_io_pp 2024-02-10T08:28:16.759 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54256 2024-02-10T08:28:16.759 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_io_pp on pid 54256' 2024-02-10T08:28:16.760 INFO:tasks.workunit.client.0.smithi067.stdout:test api_io_pp on pid 54256 2024-02-10T08:28:16.761 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54256 2024-02-10T08:28:16.761 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.761 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_asio 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_asio' 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_asio 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_io_pp 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-10T08:28:16.762 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-10T08:28:16.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_asio 2024-02-10T08:28:16.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54264 2024-02-10T08:28:16.763 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_asio on pid 54264' 2024-02-10T08:28:16.763 INFO:tasks.workunit.client.0.smithi067.stdout:test api_asio on pid 54264 2024-02-10T08:28:16.764 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54264 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_list 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_list' 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_list 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_asio 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-10T08:28:16.765 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_asio: /' 2024-02-10T08:28:16.766 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_list 2024-02-10T08:28:16.766 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54272 2024-02-10T08:28:16.766 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_list on pid 54272' 2024-02-10T08:28:16.766 INFO:tasks.workunit.client.0.smithi067.stdout:test api_list on pid 54272 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54272 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_lock 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_lock' 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_lock 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.768 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_list 2024-02-10T08:28:16.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_list.log 2024-02-10T08:28:16.769 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_list: /' 2024-02-10T08:28:16.770 INFO:tasks.workunit.client.0.smithi067.stdout:test api_lock on pid 54280 2024-02-10T08:28:16.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_lock 2024-02-10T08:28:16.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54280 2024-02-10T08:28:16.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_lock on pid 54280' 2024-02-10T08:28:16.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54280 2024-02-10T08:28:16.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.771 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.772 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-10T08:28:16.772 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_lock_pp 2024-02-10T08:28:16.772 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_lock_pp' 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_lock_pp 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_lock 2024-02-10T08:28:16.773 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-10T08:28:16.774 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_lock: /' 2024-02-10T08:28:16.774 INFO:tasks.workunit.client.0.smithi067.stdout:test api_lock_pp on pid 54288 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_lock_pp 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54288 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_lock_pp on pid 54288' 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54288 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-10T08:28:16.775 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_misc 2024-02-10T08:28:16.776 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_misc' 2024-02-10T08:28:16.776 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.776 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.776 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.776 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.777 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.777 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_lock_pp 2024-02-10T08:28:16.777 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_misc 2024-02-10T08:28:16.779 INFO:tasks.workunit.client.0.smithi067.stdout:test api_misc on pid 54296 2024-02-10T08:28:16.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_misc 2024-02-10T08:28:16.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54296 2024-02-10T08:28:16.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_misc on pid 54296' 2024-02-10T08:28:16.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54296 2024-02-10T08:28:16.779 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.780 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.780 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_misc_pp 2024-02-10T08:28:16.780 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_misc_pp' 2024-02-10T08:28:16.780 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-10T08:28:16.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.781 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.782 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_misc 2024-02-10T08:28:16.782 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-10T08:28:16.783 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_misc_pp 2024-02-10T08:28:16.783 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_misc_pp 2024-02-10T08:28:16.786 INFO:tasks.workunit.client.0.smithi067.stdout:test api_misc_pp on pid 54309 2024-02-10T08:28:16.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54309 2024-02-10T08:28:16.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_misc_pp on pid 54309' 2024-02-10T08:28:16.786 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54309 2024-02-10T08:28:16.787 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.787 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.787 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_tier_pp 2024-02-10T08:28:16.787 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_tier_pp' 2024-02-10T08:28:16.787 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-10T08:28:16.787 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-10T08:28:16.788 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.789 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.789 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.789 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.789 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.789 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_misc: /' 2024-02-10T08:28:16.790 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_misc_pp 2024-02-10T08:28:16.790 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_tier_pp 2024-02-10T08:28:16.791 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_tier_pp 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stdout:test api_tier_pp on pid 54320 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54320 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_tier_pp on pid 54320' 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54320 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-10T08:28:16.792 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-10T08:28:16.795 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_pool 2024-02-10T08:28:16.796 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_pool' 2024-02-10T08:28:16.796 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-10T08:28:16.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_tier_pp 2024-02-10T08:28:16.800 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-10T08:28:16.803 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_pool 2024-02-10T08:28:16.804 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.806 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_pool 2024-02-10T08:28:16.806 INFO:tasks.workunit.client.0.smithi067.stdout:test api_pool on pid 54348 2024-02-10T08:28:16.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54348 2024-02-10T08:28:16.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_pool on pid 54348' 2024-02-10T08:28:16.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54348 2024-02-10T08:28:16.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.807 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-10T08:28:16.808 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.808 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.808 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.808 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.809 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_snapshots 2024-02-10T08:28:16.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_snapshots' 2024-02-10T08:28:16.809 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-10T08:28:16.818 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_pool 2024-02-10T08:28:16.819 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-10T08:28:16.819 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_pool: /' 2024-02-10T08:28:16.822 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.825 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_snapshots 2024-02-10T08:28:16.835 INFO:tasks.workunit.client.0.smithi067.stdout:test api_snapshots on pid 54457 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_snapshots 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54457 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_snapshots on pid 54457' 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54457 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_snapshots_pp 2024-02-10T08:28:16.836 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_snapshots_pp' 2024-02-10T08:28:16.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.837 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.838 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_snapshots 2024-02-10T08:28:16.838 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-10T08:28:16.838 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-10T08:28:16.839 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_snapshots_pp 2024-02-10T08:28:16.840 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stdout:test api_snapshots_pp on pid 54479 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_snapshots_pp 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54479 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_snapshots_pp on pid 54479' 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54479 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-10T08:28:16.843 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.844 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_stat 2024-02-10T08:28:16.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.844 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_stat' 2024-02-10T08:28:16.845 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.845 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.845 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-10T08:28:16.845 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.845 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_snapshots_pp 2024-02-10T08:28:16.846 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_stat 2024-02-10T08:28:16.846 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-10T08:28:16.847 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_stat 2024-02-10T08:28:16.848 INFO:tasks.workunit.client.0.smithi067.stdout:test api_stat on pid 54497 2024-02-10T08:28:16.848 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54497 2024-02-10T08:28:16.848 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_stat on pid 54497' 2024-02-10T08:28:16.848 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54497 2024-02-10T08:28:16.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-10T08:28:16.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.849 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_stat_pp 2024-02-10T08:28:16.849 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_stat_pp' 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_stat_pp 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_stat 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-10T08:28:16.850 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.852 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_stat: /' 2024-02-10T08:28:16.853 INFO:tasks.workunit.client.0.smithi067.stdout:test api_stat_pp on pid 54506 2024-02-10T08:28:16.853 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_stat_pp 2024-02-10T08:28:16.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54506 2024-02-10T08:28:16.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_stat_pp on pid 54506' 2024-02-10T08:28:16.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54506 2024-02-10T08:28:16.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.854 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-10T08:28:16.855 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.855 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_watch_notify 2024-02-10T08:28:16.855 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.855 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.855 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.855 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_watch_notify' 2024-02-10T08:28:16.856 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_stat_pp 2024-02-10T08:28:16.856 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_watch_notify 2024-02-10T08:28:16.856 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-10T08:28:16.857 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-10T08:28:16.858 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.861 INFO:tasks.workunit.client.0.smithi067.stdout:test api_watch_notify on pid 54521 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_watch_notify 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54521 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_watch_notify on pid 54521' 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54521 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-10T08:28:16.862 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_watch_notify_pp 2024-02-10T08:28:16.863 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_watch_notify_pp' 2024-02-10T08:28:16.863 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_watch_notify_pp 2024-02-10T08:28:16.863 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.864 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.864 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.864 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.864 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.865 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_watch_notify_pp 2024-02-10T08:28:16.865 INFO:tasks.workunit.client.0.smithi067.stdout:test api_watch_notify_pp on pid 54529 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54529 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_watch_notify_pp on pid 54529' 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54529 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-10T08:28:16.866 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_watch_notify 2024-02-10T08:28:16.870 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.870 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.870 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.870 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.870 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-10T08:28:16.872 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-10T08:28:16.872 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_cmd 2024-02-10T08:28:16.872 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_cmd' 2024-02-10T08:28:16.873 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-02-10T08:28:16.873 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-10T08:28:16.878 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-10T08:28:16.881 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_cmd 2024-02-10T08:28:16.881 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.884 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_cmd 2024-02-10T08:28:16.885 INFO:tasks.workunit.client.0.smithi067.stdout:test api_cmd on pid 54574 2024-02-10T08:28:16.885 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54574 2024-02-10T08:28:16.885 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_cmd on pid 54574' 2024-02-10T08:28:16.885 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54574 2024-02-10T08:28:16.885 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.885 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.886 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-10T08:28:16.886 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_cmd_pp 2024-02-10T08:28:16.886 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_cmd_pp' 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_cmd_pp 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_cmd 2024-02-10T08:28:16.887 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_cmd: /' 2024-02-10T08:28:16.888 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_cmd_pp 2024-02-10T08:28:16.888 INFO:tasks.workunit.client.0.smithi067.stdout:test api_cmd_pp on pid 54588 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54588 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_cmd_pp on pid 54588' 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54588 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_service 2024-02-10T08:28:16.889 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_service' 2024-02-10T08:28:16.890 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-10T08:28:16.890 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_service 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_cmd_pp 2024-02-10T08:28:16.891 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-10T08:28:16.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_service 2024-02-10T08:28:16.892 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54596 2024-02-10T08:28:16.893 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_service on pid 54596' 2024-02-10T08:28:16.893 INFO:tasks.workunit.client.0.smithi067.stdout:test api_service on pid 54596 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54596 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_service_pp 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_service_pp' 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_service_pp 2024-02-10T08:28:16.894 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_service 2024-02-10T08:28:16.895 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_service: /' 2024-02-10T08:28:16.896 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-10T08:28:16.898 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_service.log 2024-02-10T08:28:16.898 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.902 INFO:tasks.workunit.client.0.smithi067.stdout:test api_service_pp on pid 54620 2024-02-10T08:28:16.902 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_service_pp 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54620 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_service_pp on pid 54620' 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54620 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_c_write_operations 2024-02-10T08:28:16.903 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_c_write_operations' 2024-02-10T08:28:16.904 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.904 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_c_write_operations 2024-02-10T08:28:16.904 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-10T08:28:16.906 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_c_write_operations 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stdout:test api_c_write_operations on pid 54631 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54631 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_c_write_operations on pid 54631' 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54631 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.907 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-10T08:28:16.908 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s api_c_read_operations 2024-02-10T08:28:16.908 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' api_c_read_operations' 2024-02-10T08:28:16.908 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-10T08:28:16.909 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-10T08:28:16.910 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo api_c_read_operations 2024-02-10T08:28:16.910 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=api_c_read_operations 2024-02-10T08:28:16.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54647 2024-02-10T08:28:16.912 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test api_c_read_operations on pid 54647' 2024-02-10T08:28:16.912 INFO:tasks.workunit.client.0.smithi067.stdout:test api_c_read_operations on pid 54647 2024-02-10T08:28:16.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54647 2024-02-10T08:28:16.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.913 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_c_read_operations 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s list_parallel 2024-02-10T08:28:16.914 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' list_parallel' 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_c_write_operations 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-10T08:28:16.915 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo list_parallel 2024-02-10T08:28:16.916 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-10T08:28:16.917 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-10T08:28:16.918 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_api_service_pp 2024-02-10T08:28:16.918 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.927 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=list_parallel 2024-02-10T08:28:16.927 INFO:tasks.workunit.client.0.smithi067.stdout:test list_parallel on pid 54679 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54679 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test list_parallel on pid 54679' 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54679 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.928 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s open_pools_parallel 2024-02-10T08:28:16.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' open_pools_parallel' 2024-02-10T08:28:16.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.929 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_list_parallel 2024-02-10T08:28:16.932 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ list_parallel: /' 2024-02-10T08:28:16.941 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-10T08:28:16.943 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo open_pools_parallel 2024-02-10T08:28:16.943 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stdout:test open_pools_parallel on pid 54715 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=open_pools_parallel 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54715 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test open_pools_parallel on pid 54715' 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54715 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-10T08:28:16.946 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.947 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-10T08:28:16.950 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.950 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.955 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_open_pools_parallel 2024-02-10T08:28:16.956 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-10T08:28:16.956 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-10T08:28:16.962 INFO:tasks.workunit.client.0.smithi067.stderr:++ printf %25s delete_pools_parallel 2024-02-10T08:28:16.962 INFO:tasks.workunit.client.0.smithi067.stderr:+ r=' delete_pools_parallel' 2024-02-10T08:28:16.963 INFO:tasks.workunit.client.0.smithi067.stderr:++ echo delete_pools_parallel 2024-02-10T08:28:16.963 INFO:tasks.workunit.client.0.smithi067.stderr:++ awk '{print $1}' 2024-02-10T08:28:16.965 INFO:tasks.workunit.client.0.smithi067.stderr:+ ff=delete_pools_parallel 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stdout:test delete_pools_parallel on pid 54749 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54749 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo 'test delete_pools_parallel on pid 54749' 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ pids[$f]=54749 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ ret=0 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' 1 -eq 1 ']' 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:28:16.966 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54232 2024-02-10T08:28:16.967 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54232 2024-02-10T08:28:16.967 INFO:tasks.workunit.client.0.smithi067.stderr:+ '[' -z '' ']' 2024-02-10T08:28:16.967 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.967 INFO:tasks.workunit.client.0.smithi067.stderr:+ case $- in 2024-02-10T08:28:16.967 INFO:tasks.workunit.client.0.smithi067.stderr:+ return 2024-02-10T08:28:16.968 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph_test_rados_delete_pools_parallel 2024-02-10T08:28:16.968 INFO:tasks.workunit.client.0.smithi067.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-10T08:28:16.969 INFO:tasks.workunit.client.0.smithi067.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-10T08:28:17.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:16 smithi134 bash[18722]: cluster 2024-02-10T08:28:16.677244+0000 mgr.y (mgr.14547) 219 : cluster [DBG] pgmap v163: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:16 smithi067 bash[17385]: cluster 2024-02-10T08:28:16.677244+0000 mgr.y (mgr.14547) 219 : cluster [DBG] pgmap v163: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:17.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:16 smithi067 bash[24488]: cluster 2024-02-10T08:28:16.677244+0000 mgr.y (mgr.14547) 219 : cluster [DBG] pgmap v163: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:28:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:17 smithi067 bash[17385]: audit 2024-02-10T08:28:16.911919+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:28:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:17] "GET /metrics HTTP/1.1" 200 172198 "" "Prometheus/2.33.4" 2024-02-10T08:28:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:17 smithi067 bash[24488]: audit 2024-02-10T08:28:16.911919+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:17 smithi134 bash[18722]: audit 2024-02-10T08:28:16.911919+0000 mon.c (mon.2) 86 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:22.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:28:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:28:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:23 smithi134 bash[18722]: audit 2024-02-10T08:28:17.799843+0000 mon.a (mon.0) 688 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:28:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:28:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:23 smithi067 bash[17385]: audit 2024-02-10T08:28:17.799843+0000 mon.a (mon.0) 688 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:23 smithi067 bash[24488]: audit 2024-02-10T08:28:17.799843+0000 mon.a (mon.0) 688 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:25 smithi134 bash[18722]: audit 2024-02-10T08:28:17.809357+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.67:0/3577352375' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:25 smithi134 bash[18722]: audit 2024-02-10T08:28:22.791424+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:25 smithi134 bash[18722]: cluster 2024-02-10T08:28:23.244643+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-10T08:28:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:25 smithi134 bash[18722]: audit 2024-02-10T08:28:23.262381+0000 mon.a (mon.0) 690 : audit [INF] from='client.? 172.21.15.67:0/1739150227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi067-54245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:25 smithi134 bash[18722]: audit 2024-02-10T08:28:23.264116+0000 mon.a (mon.0) 691 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[17385]: audit 2024-02-10T08:28:17.809357+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.67:0/3577352375' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[17385]: audit 2024-02-10T08:28:22.791424+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[17385]: cluster 2024-02-10T08:28:23.244643+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-10T08:28:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[17385]: audit 2024-02-10T08:28:23.262381+0000 mon.a (mon.0) 690 : audit [INF] from='client.? 172.21.15.67:0/1739150227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi067-54245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[17385]: audit 2024-02-10T08:28:23.264116+0000 mon.a (mon.0) 691 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[24488]: audit 2024-02-10T08:28:17.809357+0000 mon.c (mon.2) 87 : audit [INF] from='client.? 172.21.15.67:0/3577352375' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[24488]: audit 2024-02-10T08:28:22.791424+0000 mon.c (mon.2) 88 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[24488]: cluster 2024-02-10T08:28:23.244643+0000 mon.a (mon.0) 689 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-10T08:28:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[24488]: audit 2024-02-10T08:28:23.262381+0000 mon.a (mon.0) 690 : audit [INF] from='client.? 172.21.15.67:0/1739150227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi067-54245-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:25 smithi067 bash[24488]: audit 2024-02-10T08:28:23.264116+0000 mon.a (mon.0) 691 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: cluster 2024-02-10T08:28:18.677986+0000 mgr.y (mgr.14547) 220 : cluster [DBG] pgmap v165: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:27.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.288093+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.288640+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.67:0/1102737415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.289311+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.67:0/1498539570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.291300+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.291515+0000 mon.b (mon.1) 29 : audit [INF] from='client.? 172.21.15.67:0/2163696591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.291715+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.67:0/7812245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:23.296770+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.67:0/3132748482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: cluster 2024-02-10T08:28:25.493624+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:25.515533+0000 mon.a (mon.0) 693 : audit [INF] from='client.? 172.21.15.67:0/526334148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi067-54285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:25.515812+0000 mon.a (mon.0) 694 : audit [INF] from='client.? 172.21.15.67:0/2865010434' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi067-54237-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:26 smithi134 bash[18722]: audit 2024-02-10T08:28:25.516198+0000 mon.a (mon.0) 695 : audit [INF] from='client.? 172.21.15.67:0/260941961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: cluster 2024-02-10T08:28:18.677986+0000 mgr.y (mgr.14547) 220 : cluster [DBG] pgmap v165: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.288093+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.288640+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.67:0/1102737415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.289311+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.67:0/1498539570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.291300+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.291515+0000 mon.b (mon.1) 29 : audit [INF] from='client.? 172.21.15.67:0/2163696591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.291715+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.67:0/7812245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:23.296770+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.67:0/3132748482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: cluster 2024-02-10T08:28:25.493624+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:25.515533+0000 mon.a (mon.0) 693 : audit [INF] from='client.? 172.21.15.67:0/526334148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi067-54285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:25.515812+0000 mon.a (mon.0) 694 : audit [INF] from='client.? 172.21.15.67:0/2865010434' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi067-54237-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[17385]: audit 2024-02-10T08:28:25.516198+0000 mon.a (mon.0) 695 : audit [INF] from='client.? 172.21.15.67:0/260941961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: cluster 2024-02-10T08:28:18.677986+0000 mgr.y (mgr.14547) 220 : cluster [DBG] pgmap v165: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.288093+0000 mon.c (mon.2) 89 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.288640+0000 mon.b (mon.1) 28 : audit [INF] from='client.? 172.21.15.67:0/1102737415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.289311+0000 mon.c (mon.2) 90 : audit [INF] from='client.? 172.21.15.67:0/1498539570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.291300+0000 mon.c (mon.2) 91 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.291515+0000 mon.b (mon.1) 29 : audit [INF] from='client.? 172.21.15.67:0/2163696591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.291715+0000 mon.c (mon.2) 92 : audit [INF] from='client.? 172.21.15.67:0/7812245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:23.296770+0000 mon.c (mon.2) 93 : audit [INF] from='client.? 172.21.15.67:0/3132748482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: cluster 2024-02-10T08:28:25.493624+0000 mon.a (mon.0) 692 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:25.515533+0000 mon.a (mon.0) 693 : audit [INF] from='client.? 172.21.15.67:0/526334148' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi067-54285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:25.515812+0000 mon.a (mon.0) 694 : audit [INF] from='client.? 172.21.15.67:0/2865010434' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi067-54237-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:27.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:26 smithi067 bash[24488]: audit 2024-02-10T08:28:25.516198+0000 mon.a (mon.0) 695 : audit [INF] from='client.? 172.21.15.67:0/260941961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54593-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:28:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:27] "GET /metrics HTTP/1.1" 200 172198 "" "Prometheus/2.33.4" 2024-02-10T08:28:28.267 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-10T08:28:28.267 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [----------] Global test environment set-up. 2024-02-10T08:28:28.267 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-10T08:28:28.267 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-10T08:28:28.267 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-02-10T08:28:28.267 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (0 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (2 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (53 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (7 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (4 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-10T08:28:28.268 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (4 ms) 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [----------] 12 tests from AsioRados (80 ms total) 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [----------] Global test environment tear-down 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (11450 ms total) 2024-02-10T08:28:28.269 INFO:tasks.workunit.client.0.smithi067.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-10T08:28:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: cluster 2024-02-10T08:28:20.678834+0000 mgr.y (mgr.14547) 221 : cluster [DBG] pgmap v166: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:21.839249+0000 mgr.y (mgr.14547) 222 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: cluster 2024-02-10T08:28:22.680354+0000 mgr.y (mgr.14547) 223 : cluster [DBG] pgmap v167: 196 pgs: 46 creating+peering, 132 active+clean, 18 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: cluster 2024-02-10T08:28:24.681460+0000 mgr.y (mgr.14547) 224 : cluster [DBG] pgmap v169: 580 pgs: 46 creating+peering, 132 active+clean, 402 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:25.522748+0000 mon.a (mon.0) 696 : audit [INF] from='client.? 172.21.15.67:0/1739150227' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi067-54245-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:25.522854+0000 mon.a (mon.0) 697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:25.528231+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.67:0/3307918917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:25.528617+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.67:0/2861277660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:25.536357+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.67:0/316162746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:25.579367+0000 mon.b (mon.1) 32 : audit [DBG] from='client.? 172.21.15.67:0/1982974812' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: cluster 2024-02-10T08:28:26.685436+0000 mgr.y (mgr.14547) 225 : cluster [DBG] pgmap v171: 740 pgs: 512 creating+peering, 180 active+clean, 48 unknown; 5.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: cluster 2024-02-10T08:28:26.888433+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.909041+0000 mon.a (mon.0) 699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.911258+0000 mon.a (mon.0) 700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.918245+0000 mon.a (mon.0) 701 : audit [INF] from='client.? 172.21.15.67:0/526334148' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi067-54285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: cluster 2024-02-10T08:28:20.678834+0000 mgr.y (mgr.14547) 221 : cluster [DBG] pgmap v166: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:21.839249+0000 mgr.y (mgr.14547) 222 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: cluster 2024-02-10T08:28:22.680354+0000 mgr.y (mgr.14547) 223 : cluster [DBG] pgmap v167: 196 pgs: 46 creating+peering, 132 active+clean, 18 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: cluster 2024-02-10T08:28:24.681460+0000 mgr.y (mgr.14547) 224 : cluster [DBG] pgmap v169: 580 pgs: 46 creating+peering, 132 active+clean, 402 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:25.522748+0000 mon.a (mon.0) 696 : audit [INF] from='client.? 172.21.15.67:0/1739150227' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi067-54245-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:25.522854+0000 mon.a (mon.0) 697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:25.528231+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.67:0/3307918917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:25.528617+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.67:0/2861277660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:25.536357+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.67:0/316162746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:25.579367+0000 mon.b (mon.1) 32 : audit [DBG] from='client.? 172.21.15.67:0/1982974812' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: cluster 2024-02-10T08:28:26.685436+0000 mgr.y (mgr.14547) 225 : cluster [DBG] pgmap v171: 740 pgs: 512 creating+peering, 180 active+clean, 48 unknown; 5.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-10T08:28:28.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: cluster 2024-02-10T08:28:26.888433+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.909041+0000 mon.a (mon.0) 699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.911258+0000 mon.a (mon.0) 700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.918245+0000 mon.a (mon.0) 701 : audit [INF] from='client.? 172.21.15.67:0/526334148' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi067-54285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.918362+0000 mon.a (mon.0) 702 : audit [INF] from='client.? 172.21.15.67:0/2865010434' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi067-54237-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.918488+0000 mon.a (mon.0) 703 : audit [INF] from='client.? 172.21.15.67:0/260941961' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54593-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: cluster 2024-02-10T08:28:26.919237+0000 mon.a (mon.0) 704 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.920327+0000 mon.a (mon.0) 705 : audit [DBG] from='client.? 172.21.15.67:0/2791092272' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.921000+0000 mon.a (mon.0) 706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.921966+0000 mon.a (mon.0) 707 : audit [INF] from='client.? 172.21.15.67:0/3774411559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi067-54503-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.922947+0000 mon.a (mon.0) 708 : audit [INF] from='client.? 172.21.15.67:0/2945010213' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi067-54485-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.923241+0000 mon.a (mon.0) 709 : audit [INF] from='client.? 172.21.15.67:0/4116345738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi067-54509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.926511+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.927247+0000 mon.a (mon.0) 710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.927624+0000 mon.a (mon.0) 711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.927949+0000 mon.a (mon.0) 712 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.928553+0000 mon.a (mon.0) 713 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.928860+0000 mon.a (mon.0) 714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.918362+0000 mon.a (mon.0) 702 : audit [INF] from='client.? 172.21.15.67:0/2865010434' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi067-54237-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.918488+0000 mon.a (mon.0) 703 : audit [INF] from='client.? 172.21.15.67:0/260941961' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54593-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: cluster 2024-02-10T08:28:26.919237+0000 mon.a (mon.0) 704 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.920327+0000 mon.a (mon.0) 705 : audit [DBG] from='client.? 172.21.15.67:0/2791092272' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.921000+0000 mon.a (mon.0) 706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.921966+0000 mon.a (mon.0) 707 : audit [INF] from='client.? 172.21.15.67:0/3774411559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi067-54503-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.922947+0000 mon.a (mon.0) 708 : audit [INF] from='client.? 172.21.15.67:0/2945010213' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi067-54485-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.923241+0000 mon.a (mon.0) 709 : audit [INF] from='client.? 172.21.15.67:0/4116345738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi067-54509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.926511+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.927247+0000 mon.a (mon.0) 710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.927624+0000 mon.a (mon.0) 711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.927949+0000 mon.a (mon.0) 712 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.928553+0000 mon.a (mon.0) 713 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.928860+0000 mon.a (mon.0) 714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.930734+0000 mon.a (mon.0) 715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:26.931245+0000 mon.a (mon.0) 716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:27.393133+0000 mon.a (mon.0) 717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:27.401150+0000 mon.a (mon.0) 718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:28.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[17385]: audit 2024-02-10T08:28:27.889420+0000 mon.a (mon.0) 719 : audit [DBG] from='client.? 172.21.15.67:0/2480399595' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:28.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.930734+0000 mon.a (mon.0) 715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:26.931245+0000 mon.a (mon.0) 716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:27.393133+0000 mon.a (mon.0) 717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:28.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:27.401150+0000 mon.a (mon.0) 718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:28.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:28 smithi067 bash[24488]: audit 2024-02-10T08:28:27.889420+0000 mon.a (mon.0) 719 : audit [DBG] from='client.? 172.21.15.67:0/2480399595' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:28.958 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-10T08:28:28.958 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] Global test environment set-up. 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.ListObjects (45 ms) 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (40 ms) 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset foo1,foo2,foo3 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo1 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo2 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo3 2024-02-10T08:28:28.959 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset foo1,foo4,foo5 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo4 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo5 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo1 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset foo6,foo7 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo7 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo6 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns1:foo4 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns1:foo5 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns2:foo7 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns2:foo6 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns1:foo1 2024-02-10T08:28:28.960 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: :foo1 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: :foo2 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: :foo3 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (102 ms) 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 1 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 10 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 13 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 7 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 14 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 0 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 15 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 11 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 5 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 8 0 2024-02-10T08:28:28.961 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 6 0 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 3 0 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 4 0 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 12 0 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 9 0 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 2 0 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (98 ms) 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: x cursor=MIN 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=1 cursor=12:02547ec2:::1:head 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=10 cursor=12:52ea6a34:::10:head 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=13 cursor=12:566253c9:::13:head 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=7 cursor=12:5c6b0b28:::7:head 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=14 cursor=12:62a1935d:::14:head 2024-02-10T08:28:28.962 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=0 cursor=12:6cac518f:::0:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=15 cursor=12:863748b0:::15:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=11 cursor=12:89d3ae78:::11:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=5 cursor=12:b29083e3:::5:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=8 cursor=12:bd63b0f1:::8:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=6 cursor=12:c4fdafeb:::6:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=3 cursor=12:cfc208b3:::3:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=4 cursor=12:d83876eb:::4:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=12 cursor=12:de5d7c5f:::12:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=9 cursor=12:e960b815:::9:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > oid=2 cursor=12:f905c69b:::2:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=1 cursor=12:02547ec2:::1:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-02-10T08:28:28.963 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=10 cursor=12:52ea6a34:::10:head 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=13 cursor=12:566253c9:::13:head 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:566253c9:::13:head 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=7 cursor=12:5c6b0b28:::7:head 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-02-10T08:28:28.964 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: cluster 2024-02-10T08:28:20.678834+0000 mgr.y (mgr.14547) 221 : cluster [DBG] pgmap v166: 196 pgs: 64 unknown, 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:21.839249+0000 mgr.y (mgr.14547) 222 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: cluster 2024-02-10T08:28:22.680354+0000 mgr.y (mgr.14547) 223 : cluster [DBG] pgmap v167: 196 pgs: 46 creating+peering, 132 active+clean, 18 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: cluster 2024-02-10T08:28:24.681460+0000 mgr.y (mgr.14547) 224 : cluster [DBG] pgmap v169: 580 pgs: 46 creating+peering, 132 active+clean, 402 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:25.522748+0000 mon.a (mon.0) 696 : audit [INF] from='client.? 172.21.15.67:0/1739150227' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi067-54245-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:25.522854+0000 mon.a (mon.0) 697 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi067-54252-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:25.528231+0000 mon.b (mon.1) 30 : audit [INF] from='client.? 172.21.15.67:0/3307918917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:25.528617+0000 mon.b (mon.1) 31 : audit [INF] from='client.? 172.21.15.67:0/2861277660' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:25.536357+0000 mon.c (mon.2) 94 : audit [INF] from='client.? 172.21.15.67:0/316162746' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:25.579367+0000 mon.b (mon.1) 32 : audit [DBG] from='client.? 172.21.15.67:0/1982974812' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: cluster 2024-02-10T08:28:26.685436+0000 mgr.y (mgr.14547) 225 : cluster [DBG] pgmap v171: 740 pgs: 512 creating+peering, 180 active+clean, 48 unknown; 5.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 127 B/s wr, 1 op/s 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: cluster 2024-02-10T08:28:26.888433+0000 mon.a (mon.0) 698 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.909041+0000 mon.a (mon.0) 699 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.911258+0000 mon.a (mon.0) 700 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.918245+0000 mon.a (mon.0) 701 : audit [INF] from='client.? 172.21.15.67:0/526334148' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi067-54285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.918362+0000 mon.a (mon.0) 702 : audit [INF] from='client.? 172.21.15.67:0/2865010434' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi067-54237-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.918488+0000 mon.a (mon.0) 703 : audit [INF] from='client.? 172.21.15.67:0/260941961' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54593-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: cluster 2024-02-10T08:28:26.919237+0000 mon.a (mon.0) 704 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.920327+0000 mon.a (mon.0) 705 : audit [DBG] from='client.? 172.21.15.67:0/2791092272' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.921000+0000 mon.a (mon.0) 706 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:29.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.921966+0000 mon.a (mon.0) 707 : audit [INF] from='client.? 172.21.15.67:0/3774411559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi067-54503-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.922947+0000 mon.a (mon.0) 708 : audit [INF] from='client.? 172.21.15.67:0/2945010213' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi067-54485-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.923241+0000 mon.a (mon.0) 709 : audit [INF] from='client.? 172.21.15.67:0/4116345738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi067-54509-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.926511+0000 mon.c (mon.2) 95 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.927247+0000 mon.a (mon.0) 710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.927624+0000 mon.a (mon.0) 711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.927949+0000 mon.a (mon.0) 712 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.928553+0000 mon.a (mon.0) 713 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.928860+0000 mon.a (mon.0) 714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.930734+0000 mon.a (mon.0) 715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:26.931245+0000 mon.a (mon.0) 716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:27.393133+0000 mon.a (mon.0) 717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:27.401150+0000 mon.a (mon.0) 718 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:29.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:28 smithi134 bash[18722]: audit 2024-02-10T08:28:27.889420+0000 mon.a (mon.0) 719 : audit [DBG] from='client.? 172.21.15.67:0/2480399595' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:29.149 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=14 cursor=12:62a1935d:::14:head 2024-02-10T08:28:29.149 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=0 cursor=12:6cac518f:::0:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=15 cursor=12:863748b0:::15:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:863748b0:::15:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=11 cursor=12:89d3ae78:::11:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-02-10T08:28:29.150 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=5 cursor=12:b29083e3:::5:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=8 cursor=12:bd63b0f1:::8:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=6 cursor=12:c4fdafeb:::6:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=3 cursor=12:cfc208b3:::3:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=4 cursor=12:d83876eb:::4:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=12 cursor=12:de5d7c5f:::12:head 2024-02-10T08:28:29.151 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-02-10T08:28:29.152 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-02-10T08:28:29.152 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=9 cursor=12:e960b815:::9:head 2024-02-10T08:28:29.152 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:e960b815:::9:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : oid=2 cursor=12:f905c69b:::2:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:52ea6a34:::10:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:52ea6a34:::10:head expected=12:52ea6a34:::10:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:52ea6a34:::10:head -> 10 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=10 expected=10 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:566253c9:::13:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:566253c9:::13:head expected=12:566253c9:::13:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:566253c9:::13:head -> 13 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=13 expected=13 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:f905c69b:::2:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:f905c69b:::2:head expected=12:f905c69b:::2:head 2024-02-10T08:28:29.153 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:f905c69b:::2:head -> 2 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=2 expected=2 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:62a1935d:::14:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:62a1935d:::14:head expected=12:62a1935d:::14:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:62a1935d:::14:head -> 14 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=14 expected=14 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:5c6b0b28:::7:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:5c6b0b28:::7:head expected=12:5c6b0b28:::7:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:5c6b0b28:::7:head -> 7 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=7 expected=7 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:bd63b0f1:::8:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:bd63b0f1:::8:head expected=12:bd63b0f1:::8:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:bd63b0f1:::8:head -> 8 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=8 expected=8 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:b29083e3:::5:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:b29083e3:::5:head expected=12:b29083e3:::5:head 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:b29083e3:::5:head -> 5 2024-02-10T08:28:29.154 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=5 expected=5 2024-02-10T08:28:29.155 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:e960b815:::9:head 2024-02-10T08:28:29.155 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:e960b815:::9:head expected=12:e960b815:::9:head 2024-02-10T08:28:29.155 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:e960b815:::9:head -> 9 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : e api_cmd_pp: Running main() from gmock_main.cc 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (46 ms) 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (49 ms) 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (12542 ms) 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (12637 ms total) 2024-02-10T08:28:29.541 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: 2024-02-10T08:28:29.542 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-10T08:28:29.542 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (12637 ms total) 2024-02-10T08:28:29.542 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-10T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.929978+0000 mon.a (mon.0) 720 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930165+0000 mon.a (mon.0) 721 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930293+0000 mon.a (mon.0) 722 : audit [INF] from='client.? 172.21.15.67:0/3774411559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi067-54503-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930383+0000 mon.a (mon.0) 723 : audit [INF] from='client.? 172.21.15.67:0/2945010213' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi067-54485-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930456+0000 mon.a (mon.0) 724 : audit [INF] from='client.? 172.21.15.67:0/4116345738' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi067-54509-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930575+0000 mon.a (mon.0) 725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930652+0000 mon.a (mon.0) 726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930730+0000 mon.a (mon.0) 727 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930826+0000 mon.a (mon.0) 728 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.930916+0000 mon.a (mon.0) 729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.931013+0000 mon.a (mon.0) 730 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.931116+0000 mon.a (mon.0) 731 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.931349+0000 mon.a (mon.0) 732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.929978+0000 mon.a (mon.0) 720 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930165+0000 mon.a (mon.0) 721 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930293+0000 mon.a (mon.0) 722 : audit [INF] from='client.? 172.21.15.67:0/3774411559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi067-54503-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930383+0000 mon.a (mon.0) 723 : audit [INF] from='client.? 172.21.15.67:0/2945010213' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi067-54485-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930456+0000 mon.a (mon.0) 724 : audit [INF] from='client.? 172.21.15.67:0/4116345738' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi067-54509-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930575+0000 mon.a (mon.0) 725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930652+0000 mon.a (mon.0) 726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930730+0000 mon.a (mon.0) 727 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930826+0000 mon.a (mon.0) 728 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.930916+0000 mon.a (mon.0) 729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.931013+0000 mon.a (mon.0) 730 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.931116+0000 mon.a (mon.0) 731 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.931349+0000 mon.a (mon.0) 732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.931530+0000 mon.a (mon.0) 733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: cluster 2024-02-10T08:28:27.931708+0000 mon.a (mon.0) 734 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.957159+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:27.963014+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.67:0/2076130754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:28.537408+0000 mon.a (mon.0) 735 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:28.538522+0000 mon.a (mon.0) 736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:28.540370+0000 mon.a (mon.0) 737 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[17385]: audit 2024-02-10T08:28:29.040219+0000 mon.a (mon.0) 738 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.931530+0000 mon.a (mon.0) 733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: cluster 2024-02-10T08:28:27.931708+0000 mon.a (mon.0) 734 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.957159+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:29.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:27.963014+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.67:0/2076130754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:28.537408+0000 mon.a (mon.0) 735 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:29.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:28.538522+0000 mon.a (mon.0) 736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:29.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:28.540370+0000 mon.a (mon.0) 737 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:29.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:29 smithi067 bash[24488]: audit 2024-02-10T08:28:29.040219+0000 mon.a (mon.0) 738 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.929978+0000 mon.a (mon.0) 720 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930165+0000 mon.a (mon.0) 721 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi067-54526-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930293+0000 mon.a (mon.0) 722 : audit [INF] from='client.? 172.21.15.67:0/3774411559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi067-54503-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930383+0000 mon.a (mon.0) 723 : audit [INF] from='client.? 172.21.15.67:0/2945010213' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi067-54485-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930456+0000 mon.a (mon.0) 724 : audit [INF] from='client.? 172.21.15.67:0/4116345738' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi067-54509-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930575+0000 mon.a (mon.0) 725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930652+0000 mon.a (mon.0) 726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi067-54339-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930730+0000 mon.a (mon.0) 727 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930826+0000 mon.a (mon.0) 728 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi067-54461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.930916+0000 mon.a (mon.0) 729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi067-54277-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.931013+0000 mon.a (mon.0) 730 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi067-54293-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.931116+0000 mon.a (mon.0) 731 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi067-54261-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.931349+0000 mon.a (mon.0) 732 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi067-54649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.931530+0000 mon.a (mon.0) 733 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: cluster 2024-02-10T08:28:27.931708+0000 mon.a (mon.0) 734 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.957159+0000 mon.c (mon.2) 96 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:27.963014+0000 mon.c (mon.2) 97 : audit [INF] from='client.? 172.21.15.67:0/2076130754' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:28.537408+0000 mon.a (mon.0) 735 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:28.538522+0000 mon.a (mon.0) 736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:28.540370+0000 mon.a (mon.0) 737 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:30.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:29 smithi134 bash[18722]: audit 2024-02-10T08:28:29.040219+0000 mon.a (mon.0) 738 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [----------] Global test environment set-up. 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5041 ms) 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ OK ] LibRadosService.RegisterLate (24 ms) 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: cluster: 2024-02-10T08:28:30.120 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: id: 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: health: HEALTH_OK 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: services: 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: data: 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: objects: 192 objects, 5.7 KiB 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pgs: 132 active+clean 2024-02-10T08:28:30.121 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: io: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: cluster: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: id: 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: health: HEALTH_OK 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: services: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.122 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: data: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pools: 8 pools, 196 pgs 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: objects: 192 objects, 5.7 KiB 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pgs: 32.653% pgs unknown 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 132 active+clean 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 64 unknown 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: io: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: cluster: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: id: 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: health: HEALTH_OK 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: services: 2024-02-10T08:28:30.123 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: foo: 15 portals active (1 hosts, 3 zones) 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: data: 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pools: 25 pools, 740 pgs 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: objects: 193 objects, 5.8 KiB 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pgs: 6.486% pgs unknown 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 69.189% pgs not active 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 512 creating+peering 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 180 active+clean 2024-02-10T08:28:30.124 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 48 unknown 2024-02-10T08:28:30.125 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:30.324 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: starting. 2024-02-10T08:28:30.324 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: creating pool ceph_test_rados_list_parallel.smithi067-54683 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: created object 0... 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: created object 25... 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: created object 49... 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: finishing. 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_1_[54702]: shutting down. 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_2_[54703]: starting. 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_2_[54703]: listing objects. 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_2_[54703]: listed object 0... 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_2_[54703]: listed object 25... 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_2_[54703]: saw 50 objects 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_2_[54703]: shutting down. 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: starting. 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: creating pool ceph_test_rados_list_parallel.smithi067-54683 2024-02-10T08:28:30.325 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: created object 0... 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: created object 25... 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: created object 49... 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: finishing. 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_3_[55475]: shutting down. 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_5_[55480]: starting. 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_5_[55480]: removed 25 objects... 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_5_[55480]: removed half of the objects 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_5_[55480]: removed 50 objects... 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_5_[55480]: removed 50 objects 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_5_[55480]: shutting down. 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_4_[55476]: starting. 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_4_[55476]: listing objects. 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_4_[55476]: saw 0 objects 2024-02-10T08:28:30.326 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_4_[55476]: shutting down. 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: starting. 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: creating pool ceph_test_rados_list_parallel.smithi067-54683 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: created object 0... 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: created object 25... 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: created object 49... 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: finishing. 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_6_[55540]: shutting down. 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: starting. 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: listing objects. 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: listed object 0... 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: listed object 25... 2024-02-10T08:28:30.327 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: listed object 50... 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: saw 56 objects 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_7_[55541]: shutting down. 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_8_[55542]: starting. 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_8_[55542]: added 25 objects... 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_8_[55542]: added half of the objects 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_8_[55542]: added 50 objects... 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_8_[55542]: added 50 objects 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_8_[55542]: shutting down. 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:30.328 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: starting. 2024-02-10T08:28:31.155 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: creating pool ceph_t api_c_read_operations: Running main() from gmock_main.cc 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (1 ms) 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (9 ms) 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (6 ms) 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (6 ms) 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-10T08:28:31.156 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (22 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (9 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (7 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (8 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (6 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (7 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (6 ms) 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-10T08:28:31.157 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (16 ms) 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (16 ms) 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (11 ms) 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (143 ms total) 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (14230 ms total) 2024-02-10T08:28:31.158 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-02-10T08:28:31.411 INFO:tasks.workunit.client.0.smithi067.stdout:est_rados_list_parallel.smithi067-54683 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: created object 0... 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: created object 25... 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: created object 49... 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: finishing. 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_9_[55589]: shutting down. 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: starting. 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: listing objects. 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: listed object 0... 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: listed object 25... 2024-02-10T08:28:31.412 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: listed object 50... 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: listed object 75... 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: saw 97 objects 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_10_[55590]: shutting down. 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_13_[55593]: starting. 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_13_[55593]: removed 25 objects... 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_13_[55593]: removed half of the objects 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_13_[55593]: removed 50 objects... 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_13_[55593]: removed 50 objects 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_13_[55593]: shutting down. 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.413 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_11_[55591]: starting. 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_11_[55591]: added 25 objects... 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_11_[55591]: added half of the objects 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_11_[55591]: added 50 objects... 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_11_[55591]: added 50 objects 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_11_[55591]: shutting down. 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_12_[55592]: starting. 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_12_[55592]: added 25 objects... 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_12_[55592]: added half of the objects 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_12_[55592]: added 50 objects... 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_12_[55592]: added 50 objects 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_12_[55592]: shutting down. 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.414 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: starting. 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: creating pool ceph_test_rados_list_parallel.smithi067-54683 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: created object 0... 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: created object 25... 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: created object 49... 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: finishing. 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_14_[55770]: shutting down. 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: starting. 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listing objects. 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listed object 0... 2024-02-10T08:28:31.415 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listed object 25... 2024-02-10T08:28:31.416 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listed object 50... 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: cluster 2024-02-10T08:28:28.686705+0000 mgr.y (mgr.14547) 226 : cluster [DBG] pgmap v174: 804 pgs: 464 creating+peering, 132 active+clean, 208 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:29.264697+0000 mon.c (mon.2) 98 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:29.530417+0000 mon.a (mon.0) 739 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: cluster 2024-02-10T08:28:29.532164+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:29.532588+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.67:0/89287961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:29.548054+0000 mon.a (mon.0) 741 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:30.095477+0000 mon.a (mon.0) 742 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:30.095876+0000 mon.a (mon.0) 743 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: cluster 2024-02-10T08:28:30.096588+0000 mon.a (mon.0) 744 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:30.109672+0000 mon.a (mon.0) 745 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:30.111337+0000 mon.a (mon.0) 746 : audit [DBG] from='client.? 172.21.15.67:0/2207430617' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:31.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:30.113993+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[18722]: audit 2024-02-10T08:28:30.118639+0000 mon.a (mon.0) 747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: cluster 2024-02-10T08:28:28.686705+0000 mgr.y (mgr.14547) 226 : cluster [DBG] pgmap v174: 804 pgs: 464 creating+peering, 132 active+clean, 208 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-02-10T08:28:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:29.264697+0000 mon.c (mon.2) 98 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:29.530417+0000 mon.a (mon.0) 739 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: cluster 2024-02-10T08:28:29.532164+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-10T08:28:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:29.532588+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.67:0/89287961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:29.548054+0000 mon.a (mon.0) 741 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:30.095477+0000 mon.a (mon.0) 742 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:30.095876+0000 mon.a (mon.0) 743 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: cluster 2024-02-10T08:28:30.096588+0000 mon.a (mon.0) 744 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:30.109672+0000 mon.a (mon.0) 745 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:30.111337+0000 mon.a (mon.0) 746 : audit [DBG] from='client.? 172.21.15.67:0/2207430617' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:30.113993+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[17385]: audit 2024-02-10T08:28:30.118639+0000 mon.a (mon.0) 747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: cluster 2024-02-10T08:28:28.686705+0000 mgr.y (mgr.14547) 226 : cluster [DBG] pgmap v174: 804 pgs: 464 creating+peering, 132 active+clean, 208 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 945 B/s rd, 0 op/s 2024-02-10T08:28:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:29.264697+0000 mon.c (mon.2) 98 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:29.530417+0000 mon.a (mon.0) 739 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: cluster 2024-02-10T08:28:29.532164+0000 mon.a (mon.0) 740 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:29.532588+0000 mon.b (mon.1) 33 : audit [INF] from='client.? 172.21.15.67:0/89287961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:29.548054+0000 mon.a (mon.0) 741 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:30.095477+0000 mon.a (mon.0) 742 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:30.095876+0000 mon.a (mon.0) 743 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: cluster 2024-02-10T08:28:30.096588+0000 mon.a (mon.0) 744 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:30.109672+0000 mon.a (mon.0) 745 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:30.111337+0000 mon.a (mon.0) 746 : audit [DBG] from='client.? 172.21.15.67:0/2207430617' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:30.113993+0000 mon.c (mon.2) 99 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:31.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:31 smithi067 bash[24488]: audit 2024-02-10T08:28:30.118639+0000 mon.a (mon.0) 747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:32.161 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:28:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:28:32.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:32 smithi134 bash[18722]: audit 2024-02-10T08:28:30.215496+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:32 smithi134 bash[18722]: audit 2024-02-10T08:28:30.266788+0000 mon.c (mon.2) 100 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:32 smithi134 bash[18722]: cluster 2024-02-10T08:28:30.687669+0000 mgr.y (mgr.14547) 227 : cluster [DBG] pgmap v177: 748 pgs: 379 creating+peering, 132 active+clean, 237 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:32 smithi067 bash[17385]: audit 2024-02-10T08:28:30.215496+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:32 smithi067 bash[17385]: audit 2024-02-10T08:28:30.266788+0000 mon.c (mon.2) 100 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:32 smithi067 bash[17385]: cluster 2024-02-10T08:28:30.687669+0000 mgr.y (mgr.14547) 227 : cluster [DBG] pgmap v177: 748 pgs: 379 creating+peering, 132 active+clean, 237 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:32 smithi067 bash[24488]: audit 2024-02-10T08:28:30.215496+0000 mon.b (mon.1) 34 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:32 smithi067 bash[24488]: audit 2024-02-10T08:28:30.266788+0000 mon.c (mon.2) 100 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:32 smithi067 bash[24488]: cluster 2024-02-10T08:28:30.687669+0000 mgr.y (mgr.14547) 227 : cluster [DBG] pgmap v177: 748 pgs: 379 creating+peering, 132 active+clean, 237 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listed object api_cmd: Running main() from gmock_main.cc 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (10025 ms) 2024-02-10T08:28:33.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (482 ms) 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (4266 ms) 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:29.530417+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54585-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:29.532588+0000 mon.b [INF] from='client.? 172.21.15.67:0/89287961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:29.548054+0000 mon.a [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:30.095477+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi067-54538-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:30.095876+0000 mon.a [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi067-54252-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:30.109672+0000 mon.a [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:30.113993+0000 mon.c [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:30.118639+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:30.215496+0000 mon.b [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:31.141452+0000 mon.c [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:31.667146+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:31.669984+0000 mon.b [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:31.692959+0000 mon.c [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:31.693198+0000 client.admin [INF] onexx 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.153289+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.154018+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.168297+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.177711+0000 mon.a [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.182038+0000 mon.c [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.185598+0000 mon.a [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.185751+0000 mon.b [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.194683+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.267014+0000 mon.b [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.615991+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:33.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.617127+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:33.184 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.618115+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:33.184 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.619013+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:33.184 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.619929+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:33.184 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.621369+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:33.184 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.622328+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:28:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:31.141452+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:31.268694+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:31.667146+0000 mon.a (mon.0) 748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:31.669984+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:31.692959+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: cluster 2024-02-10T08:28:31.693198+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: cluster 2024-02-10T08:28:32.144456+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.153289+0000 mon.a (mon.0) 750 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.154018+0000 mon.a (mon.0) 751 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.168297+0000 mon.a (mon.0) 752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.177711+0000 mon.a (mon.0) 753 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: cluster 2024-02-10T08:28:32.178717+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.182038+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.183693+0000 mon.a (mon.0) 755 : audit [DBG] from='client.? 172.21.15.67:0/1946645339' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.185598+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.185751+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.194683+0000 mon.a (mon.0) 757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.267014+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.272227+0000 mon.c (mon.2) 105 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.615991+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.617127+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.618115+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.619013+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.619929+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.621369+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.622328+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.623222+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.624010+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.624758+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.704245+0000 mon.a (mon.0) 758 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.704997+0000 mon.a (mon.0) 759 : audit [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.705447+0000 mon.a (mon.0) 760 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:33 smithi134 bash[18722]: audit 2024-02-10T08:28:32.708306+0000 mon.a (mon.0) 761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:31.141452+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:31.268694+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:31.667146+0000 mon.a (mon.0) 748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:31.669984+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:31.692959+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: cluster 2024-02-10T08:28:31.693198+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: cluster 2024-02-10T08:28:32.144456+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.153289+0000 mon.a (mon.0) 750 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.154018+0000 mon.a (mon.0) 751 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.168297+0000 mon.a (mon.0) 752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.177711+0000 mon.a (mon.0) 753 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: cluster 2024-02-10T08:28:32.178717+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.182038+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.183693+0000 mon.a (mon.0) 755 : audit [DBG] from='client.? 172.21.15.67:0/1946645339' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.185598+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.185751+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:31.141452+0000 mon.c (mon.2) 101 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:31.268694+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:31.667146+0000 mon.a (mon.0) 748 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:31.669984+0000 mon.b (mon.1) 35 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:31.692959+0000 mon.c (mon.2) 103 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: cluster 2024-02-10T08:28:31.693198+0000 client.admin (client.?) 0 : cluster [INF] onexx 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: cluster 2024-02-10T08:28:32.144456+0000 mon.a (mon.0) 749 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.153289+0000 mon.a (mon.0) 750 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.154018+0000 mon.a (mon.0) 751 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.168297+0000 mon.a (mon.0) 752 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.177711+0000 mon.a (mon.0) 753 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi067-54252-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: cluster 2024-02-10T08:28:32.178717+0000 mon.a (mon.0) 754 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.182038+0000 mon.c (mon.2) 104 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.183693+0000 mon.a (mon.0) 755 : audit [DBG] from='client.? 172.21.15.67:0/1946645339' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.185598+0000 mon.a (mon.0) 756 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.185751+0000 mon.b (mon.1) 36 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.194683+0000 mon.a (mon.0) 757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.267014+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.272227+0000 mon.c (mon.2) 105 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.615991+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:33.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.617127+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.618115+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.619013+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.619929+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.621369+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.622328+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.623222+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.624010+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.624758+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.704245+0000 mon.a (mon.0) 758 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.704997+0000 mon.a (mon.0) 759 : audit [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.705447+0000 mon.a (mon.0) 760 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[24488]: audit 2024-02-10T08:28:32.708306+0000 mon.a (mon.0) 761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:33.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.194683+0000 mon.a (mon.0) 757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.267014+0000 mon.b (mon.1) 37 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.272227+0000 mon.c (mon.2) 105 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.615991+0000 mon.c (mon.2) 106 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.617127+0000 mon.c (mon.2) 107 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.618115+0000 mon.c (mon.2) 108 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.619013+0000 mon.c (mon.2) 109 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.619929+0000 mon.c (mon.2) 110 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.621369+0000 mon.c (mon.2) 111 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.622328+0000 mon.c (mon.2) 112 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.623222+0000 mon.c (mon.2) 113 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.624010+0000 mon.c (mon.2) 114 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.624758+0000 mon.c (mon.2) 115 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.704245+0000 mon.a (mon.0) 758 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.704997+0000 mon.a (mon.0) 759 : audit [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.705447+0000 mon.a (mon.0) 760 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:33.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:33 smithi067 bash[17385]: audit 2024-02-10T08:28:32.708306+0000 mon.a (mon.0) 761 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-0 75... 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listed object 100... 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: listed object 125... 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: saw 150 objects 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_15_[55771]: shutting down. 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_16_[55772]: starting. 2024-02-10T08:28:34.216 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_16_[55772]: added 25 objects... 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_16_[55772]: added half of the objects 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_16_[55772]: added 50 objects... 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_16_[55772]: added 50 objects 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: process_16_[55772]: shutting down. 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******************************* 2024-02-10T08:28:34.217 INFO:tasks.workunit.client.0.smithi067.stdout: list_parallel: ******* SUCCESS ********** 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: starting. 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: creating pool ceph_test_rados_delete_pools_parallel.smithi067-54765 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: created object 0... 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: created object 25... 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: created object 49... 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: finishing. 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_1_[54785]: shutting down. 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_2_[54786]: starting. 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_2_[54786]: deleting pool ceph_test_rados_delete_pools_parallel.smithi067-54765 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_2_[54786]: shutting down. 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: ******************************* 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: starting. 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: creating pool ceph_test_rados_delete_pools_parallel.smithi067-54765 2024-02-10T08:28:34.222 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: created object 0... 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: created object 25... 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: created object 49... 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: finishing. 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_3_[55684]: shutting down. 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: ******************************* 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_5_[55686]: starting. 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_5_[55686]: listing objects. 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_5_[55686]: listed object 0... 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_5_[55686]: listed object 25... 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_5_[55686]: saw 50 objects 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_5_[55686]: shutting down. 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: ******************************* 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_4_[55685]: starting. 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_4_[55685]: deleting pool ceph_test_rados_delete_pools_parallel.smithi067-54765 2024-02-10T08:28:34.223 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: process_4_[55685]: shutting down. 2024-02-10T08:28:34.224 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: ******************************* 2024-02-10T08:28:34.224 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: ******************************* 2024-02-10T08:28:34.224 INFO:tasks.workunit.client.0.smithi067.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-10T08:28:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:31.847452+0000 mgr.y (mgr.14547) 228 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.616305+0000 mgr.y (mgr.14547) 229 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.617395+0000 mgr.y (mgr.14547) 230 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.618351+0000 mgr.y (mgr.14547) 231 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.619252+0000 mgr.y (mgr.14547) 232 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.620591+0000 mgr.y (mgr.14547) 233 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.621594+0000 mgr.y (mgr.14547) 234 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.622555+0000 mgr.y (mgr.14547) 235 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.623408+0000 mgr.y (mgr.14547) 236 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.624188+0000 mgr.y (mgr.14547) 237 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.624937+0000 mgr.y (mgr.14547) 238 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:32.688798+0000 mgr.y (mgr.14547) 239 : cluster [DBG] pgmap v180: 628 pgs: 86 creating+peering, 490 active+clean, 52 unknown; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 135 KiB/s wr, 639 op/s 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:32.710018+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:33.045241+0000 osd.0 (osd.0) 3 : cluster [DBG] 16.0 deep-scrub starts 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:33.048982+0000 osd.0 (osd.0) 4 : cluster [DBG] 16.0 deep-scrub ok 2024-02-10T08:28:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.175572+0000 mon.a (mon.0) 762 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.184215+0000 mon.a (mon.0) 763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.185409+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.185737+0000 mon.a (mon.0) 764 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:33.185908+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.193541+0000 mon.a (mon.0) 765 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.193961+0000 mon.a (mon.0) 766 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.194511+0000 mon.a (mon.0) 767 : audit [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.208428+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.214016+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.215710+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.274540+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:33.696226+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.705367+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T08:28:34.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.706437+0000 mon.a (mon.0) 770 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.707262+0000 mon.a (mon.0) 771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.707702+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.711468+0000 mon.a (mon.0) 772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.713061+0000 mon.a (mon.0) 773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.713379+0000 mon.a (mon.0) 774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:33.714259+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:33.754564+0000 osd.2 (osd.2) 3 : cluster [DBG] 16.1 deep-scrub starts 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: cluster 2024-02-10T08:28:33.758814+0000 osd.2 (osd.2) 4 : cluster [DBG] 16.1 deep-scrub ok 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:34.173258+0000 mon.a (mon.0) 775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:34.174398+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.530 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:34 smithi134 bash[18722]: audit 2024-02-10T08:28:34.175172+0000 mon.a (mon.0) 776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:31.847452+0000 mgr.y (mgr.14547) 228 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.616305+0000 mgr.y (mgr.14547) 229 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.617395+0000 mgr.y (mgr.14547) 230 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.618351+0000 mgr.y (mgr.14547) 231 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.619252+0000 mgr.y (mgr.14547) 232 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.620591+0000 mgr.y (mgr.14547) 233 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.621594+0000 mgr.y (mgr.14547) 234 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.622555+0000 mgr.y (mgr.14547) 235 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.623408+0000 mgr.y (mgr.14547) 236 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.624188+0000 mgr.y (mgr.14547) 237 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.624937+0000 mgr.y (mgr.14547) 238 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:32.688798+0000 mgr.y (mgr.14547) 239 : cluster [DBG] pgmap v180: 628 pgs: 86 creating+peering, 490 active+clean, 52 unknown; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 135 KiB/s wr, 639 op/s 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:32.710018+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:33.045241+0000 osd.0 (osd.0) 3 : cluster [DBG] 16.0 deep-scrub starts 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:33.048982+0000 osd.0 (osd.0) 4 : cluster [DBG] 16.0 deep-scrub ok 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.175572+0000 mon.a (mon.0) 762 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.184215+0000 mon.a (mon.0) 763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T08:28:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.185409+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:31.847452+0000 mgr.y (mgr.14547) 228 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.616305+0000 mgr.y (mgr.14547) 229 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.617395+0000 mgr.y (mgr.14547) 230 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.618351+0000 mgr.y (mgr.14547) 231 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.619252+0000 mgr.y (mgr.14547) 232 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.620591+0000 mgr.y (mgr.14547) 233 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.621594+0000 mgr.y (mgr.14547) 234 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.622555+0000 mgr.y (mgr.14547) 235 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.623408+0000 mgr.y (mgr.14547) 236 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.624188+0000 mgr.y (mgr.14547) 237 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.624937+0000 mgr.y (mgr.14547) 238 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:32.688798+0000 mgr.y (mgr.14547) 239 : cluster [DBG] pgmap v180: 628 pgs: 86 creating+peering, 490 active+clean, 52 unknown; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 135 KiB/s wr, 639 op/s 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:32.710018+0000 mon.b (mon.1) 38 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:33.045241+0000 osd.0 (osd.0) 3 : cluster [DBG] 16.0 deep-scrub starts 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:33.048982+0000 osd.0 (osd.0) 4 : cluster [DBG] 16.0 deep-scrub ok 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.175572+0000 mon.a (mon.0) 762 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.184215+0000 mon.a (mon.0) 763 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.185409+0000 mon.c (mon.2) 116 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T08:28:34.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.185737+0000 mon.a (mon.0) 764 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:33.185908+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.193541+0000 mon.a (mon.0) 765 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.193961+0000 mon.a (mon.0) 766 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.194511+0000 mon.a (mon.0) 767 : audit [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.208428+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.214016+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.215710+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.274540+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:33.696226+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.705367+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.706437+0000 mon.a (mon.0) 770 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.707262+0000 mon.a (mon.0) 771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.707702+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.711468+0000 mon.a (mon.0) 772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.713061+0000 mon.a (mon.0) 773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.713379+0000 mon.a (mon.0) 774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:33.714259+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:33.754564+0000 osd.2 (osd.2) 3 : cluster [DBG] 16.1 deep-scrub starts 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: cluster 2024-02-10T08:28:33.758814+0000 osd.2 (osd.2) 4 : cluster [DBG] 16.1 deep-scrub ok 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:34.173258+0000 mon.a (mon.0) 775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:34.174398+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[17385]: audit 2024-02-10T08:28:34.175172+0000 mon.a (mon.0) 776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.185737+0000 mon.a (mon.0) 764 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:33.185908+0000 client.admin (client.?) 0 : cluster [INF] twoxx 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.193541+0000 mon.a (mon.0) 765 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi067-54237-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.193961+0000 mon.a (mon.0) 766 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.194511+0000 mon.a (mon.0) 767 : audit [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.208428+0000 mon.c (mon.2) 117 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.214016+0000 mon.c (mon.2) 118 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.215710+0000 mon.c (mon.2) 119 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.274540+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:33.696226+0000 mon.a (mon.0) 768 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.705367+0000 mon.a (mon.0) 769 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.706437+0000 mon.a (mon.0) 770 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.707262+0000 mon.a (mon.0) 771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.707702+0000 mon.c (mon.2) 121 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.711468+0000 mon.a (mon.0) 772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:34.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.713061+0000 mon.a (mon.0) 773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.713379+0000 mon.a (mon.0) 774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:33.714259+0000 mon.b (mon.1) 39 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:33.754564+0000 osd.2 (osd.2) 3 : cluster [DBG] 16.1 deep-scrub starts 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: cluster 2024-02-10T08:28:33.758814+0000 osd.2 (osd.2) 4 : cluster [DBG] 16.1 deep-scrub ok 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:34.173258+0000 mon.a (mon.0) 775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:34.174398+0000 mon.c (mon.2) 122 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:34.627 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:34 smithi067 bash[24488]: audit 2024-02-10T08:28:34.175172+0000 mon.a (mon.0) 776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: cluster 2024-02-10T08:28:34.176304+0000 mon.a (mon.0) 777 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.189715+0000 mon.a (mon.0) 778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T08:28:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.193780+0000 mon.a (mon.0) 779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.202386+0000 mon.a (mon.0) 780 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.202718+0000 mon.a (mon.0) 781 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.202878+0000 mon.a (mon.0) 782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.203119+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.203427+0000 mon.a (mon.0) 784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: cluster 2024-02-10T08:28:34.204282+0000 mon.a (mon.0) 785 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.207756+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.208327+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.208747+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.210903+0000 mon.a (mon.0) 786 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.211450+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.213156+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.213491+0000 mon.a (mon.0) 787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:35.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.215307+0000 mon.a (mon.0) 788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.215737+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.215887+0000 mon.a (mon.0) 789 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.216117+0000 mon.a (mon.0) 790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.216311+0000 mon.a (mon.0) 791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.216812+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.67:0/3679478398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.217076+0000 mon.a (mon.0) 792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.218996+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.219959+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.220254+0000 mon.a (mon.0) 794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: audit 2024-02-10T08:28:34.279087+0000 mon.c (mon.2) 128 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:35.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:35 smithi134 bash[18722]: cluster 2024-02-10T08:28:34.689890+0000 mgr.y (mgr.14547) 240 : cluster [DBG] pgmap v183: 620 pgs: 48 creating+peering, 397 active+clean, 175 unknown; 9.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 501 op/s 2024-02-10T08:28:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: cluster 2024-02-10T08:28:34.176304+0000 mon.a (mon.0) 777 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.189715+0000 mon.a (mon.0) 778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.193780+0000 mon.a (mon.0) 779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.202386+0000 mon.a (mon.0) 780 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.202718+0000 mon.a (mon.0) 781 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.202878+0000 mon.a (mon.0) 782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.203119+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.203427+0000 mon.a (mon.0) 784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: cluster 2024-02-10T08:28:34.204282+0000 mon.a (mon.0) 785 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.207756+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.208327+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.208747+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.210903+0000 mon.a (mon.0) 786 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: cluster 2024-02-10T08:28:34.176304+0000 mon.a (mon.0) 777 : cluster [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.189715+0000 mon.a (mon.0) 778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.193780+0000 mon.a (mon.0) 779 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.202386+0000 mon.a (mon.0) 780 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.202718+0000 mon.a (mon.0) 781 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.202878+0000 mon.a (mon.0) 782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.203119+0000 mon.a (mon.0) 783 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.203427+0000 mon.a (mon.0) 784 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: cluster 2024-02-10T08:28:34.204282+0000 mon.a (mon.0) 785 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.207756+0000 mon.b (mon.1) 40 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:35.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.208327+0000 mon.b (mon.1) 41 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.208747+0000 mon.b (mon.1) 42 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.210903+0000 mon.a (mon.0) 786 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.211450+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.213156+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.213491+0000 mon.a (mon.0) 787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.215307+0000 mon.a (mon.0) 788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.215737+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.215887+0000 mon.a (mon.0) 789 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.216117+0000 mon.a (mon.0) 790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.216311+0000 mon.a (mon.0) 791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.216812+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.67:0/3679478398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.217076+0000 mon.a (mon.0) 792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.218996+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.219959+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.220254+0000 mon.a (mon.0) 794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.211450+0000 mon.c (mon.2) 123 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.213156+0000 mon.c (mon.2) 124 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.213491+0000 mon.a (mon.0) 787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.215307+0000 mon.a (mon.0) 788 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.215737+0000 mon.c (mon.2) 125 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.215887+0000 mon.a (mon.0) 789 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.216117+0000 mon.a (mon.0) 790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.216311+0000 mon.a (mon.0) 791 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.216812+0000 mon.c (mon.2) 126 : audit [INF] from='client.? 172.21.15.67:0/3679478398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.217076+0000 mon.a (mon.0) 792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.218996+0000 mon.c (mon.2) 127 : audit [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:35.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.219959+0000 mon.a (mon.0) 793 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:35.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.220254+0000 mon.a (mon.0) 794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:35.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: audit 2024-02-10T08:28:34.279087+0000 mon.c (mon.2) 128 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:35.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[17385]: cluster 2024-02-10T08:28:34.689890+0000 mgr.y (mgr.14547) 240 : cluster [DBG] pgmap v183: 620 pgs: 48 creating+peering, 397 active+clean, 175 unknown; 9.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 501 op/s 2024-02-10T08:28:35.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: audit 2024-02-10T08:28:34.279087+0000 mon.c (mon.2) 128 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:35.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:35 smithi067 bash[24488]: cluster 2024-02-10T08:28:34.689890+0000 mgr.y (mgr.14547) 240 : cluster [DBG] pgmap v183: 620 pgs: 48 creating+peering, 397 active+clean, 175 unknown; 9.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 501 op/s 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout:2-10T08:28:32.623222+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.624010+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.624758+0000 mon.c [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.704245+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.704997+0000 mon.a [INF] from='client.? 172.21.15.67:0/2728955293' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.705447+0000 mon.a [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:32.708306+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:36.194 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.176304+0000 mon.a [WRN] Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.189715+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.193780+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.202386+0000 mon.a [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.202718+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.202878+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi067-54245-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.203119+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi067-54503-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.203427+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi067-54285-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.207756+0000 mon.b [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.208327+0000 mon.b [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:36.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.208747+0000 mon.b [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.210903+0000 mon.a [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.211450+0000 mon.c [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.213156+0000 mon.c [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.213491+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.215307+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.215737+0000 mon.c [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.215887+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.216117+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.216311+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.216812+0000 mon.c [INF] from='client.? 172.21.15.67:0/3679478398' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.217076+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:36.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.218996+0000 mon.c [INF] from='client.? 172.21.15.67:0/795941909' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.219959+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","po open_pools_parallel: process_1_[54750]: starting. 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_1_[54750]: creating pool ceph_test_rados_open_pools_parallel.smithi067-54729 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_1_[54750]: created object 0... 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_1_[54750]: created object 25... 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_1_[54750]: created object 49... 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_1_[54750]: finishing. 2024-02-10T08:28:37.070 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_1_[54750]: shutting down. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_2_[54755]: starting. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_2_[54755]: rados_pool_create. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_2_[54755]: rados_ioctx_create. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_2_[54755]: shutting down. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: ******************************* 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: starting. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: creating pool ceph_test_rados_open_pools_parallel.smithi067-54729 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: created object 0... 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: created object 25... 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: created object 49... 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: finishing. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_3_[55847]: shutting down. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: ******************************* 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_4_[55848]: starting. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_4_[55848]: rados_pool_create. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_4_[55848]: rados_ioctx_create. 2024-02-10T08:28:37.071 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: process_4_[55848]: shutting down. 2024-02-10T08:28:37.072 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: ******************************* 2024-02-10T08:28:37.072 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: ******************************* 2024-02-10T08:28:37.072 INFO:tasks.workunit.client.0.smithi067.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-10T08:28:38.019 INFO:tasks.workunit.client.0.smithi067.stdout:ol": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:38.019 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:34.220254+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-10T08:28:38.019 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.193677+0000 mon.c [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.194139+0000 client.admin [INF] threexx 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.592358+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.592567+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.592721+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.593782+0000 mon.c [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.594308+0000 mon.b [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:36.595836+0000 mon.b [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.042754+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.042991+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.057781+0000 mon.a [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.059153+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:38.020 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.059244+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:35.281002+0000 mon.c (mon.2) 129 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:35.558539+0000 osd.5 (osd.5) 3 : cluster [DBG] 16.3 deep-scrub starts 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:35.611170+0000 osd.5 (osd.5) 4 : cluster [DBG] 16.3 deep-scrub ok 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.193677+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:36.194139+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.284468+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:36.440672+0000 osd.6 (osd.6) 3 : cluster [DBG] 16.6 deep-scrub starts 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:36.482716+0000 osd.6 (osd.6) 4 : cluster [DBG] 16.6 deep-scrub ok 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.592358+0000 mon.a (mon.0) 795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.592567+0000 mon.a (mon.0) 796 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.592721+0000 mon.a (mon.0) 797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.593782+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.594308+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:36.595836+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:36.693864+0000 mgr.y (mgr.14547) 241 : cluster [DBG] pgmap v185: 548 pgs: 23 creating+peering, 516 active+clean, 9 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 20 op/s 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.042754+0000 mon.a (mon.0) 798 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.042991+0000 mon.a (mon.0) 799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:37.043944+0000 mon.a (mon.0) 800 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.057781+0000 mon.a (mon.0) 801 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.059153+0000 mon.a (mon.0) 802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.059244+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.059388+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:38.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: cluster 2024-02-10T08:28:37.059639+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.062701+0000 mon.a (mon.0) 806 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.063693+0000 mon.a (mon.0) 807 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.071567+0000 mon.a (mon.0) 808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.071973+0000 mon.a (mon.0) 809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.079781+0000 mon.a (mon.0) 810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[17385]: audit 2024-02-10T08:28:37.080521+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:28:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:37] "GET /metrics HTTP/1.1" 200 221848 "" "Prometheus/2.33.4" 2024-02-10T08:28:38.125 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:35.281002+0000 mon.c (mon.2) 129 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:35.558539+0000 osd.5 (osd.5) 3 : cluster [DBG] 16.3 deep-scrub starts 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:35.611170+0000 osd.5 (osd.5) 4 : cluster [DBG] 16.3 deep-scrub ok 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.193677+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:36.194139+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.284468+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:36.440672+0000 osd.6 (osd.6) 3 : cluster [DBG] 16.6 deep-scrub starts 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:36.482716+0000 osd.6 (osd.6) 4 : cluster [DBG] 16.6 deep-scrub ok 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.592358+0000 mon.a (mon.0) 795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.592567+0000 mon.a (mon.0) 796 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.592721+0000 mon.a (mon.0) 797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.593782+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.594308+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:36.595836+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:36.693864+0000 mgr.y (mgr.14547) 241 : cluster [DBG] pgmap v185: 548 pgs: 23 creating+peering, 516 active+clean, 9 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 20 op/s 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.042754+0000 mon.a (mon.0) 798 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.042991+0000 mon.a (mon.0) 799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:37.043944+0000 mon.a (mon.0) 800 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.057781+0000 mon.a (mon.0) 801 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.059153+0000 mon.a (mon.0) 802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.059244+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.059388+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: cluster 2024-02-10T08:28:37.059639+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.062701+0000 mon.a (mon.0) 806 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.063693+0000 mon.a (mon.0) 807 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.071567+0000 mon.a (mon.0) 808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.071973+0000 mon.a (mon.0) 809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.079781+0000 mon.a (mon.0) 810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.128 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:38 smithi067 bash[24488]: audit 2024-02-10T08:28:37.080521+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:38.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.059388+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibR api_io_pp: Running main() from gmock_main.cc 2024-02-10T08:28:38.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-10T08:28:38.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-10T08:28:38.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-10T08:28:38.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: seed 54261 2024-02-10T08:28:38.195 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (13 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (5 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (8 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (6 ms) 2024-02-10T08:28:38.196 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (6 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (6 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (5 ms) 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-10T08:28:38.197 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (14 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (6 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (5 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (6 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (123 ms total) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1091 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (18 ms) 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-10T08:28:38.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-02-10T08:28:38.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-10T08:28:38.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (6 ms) 2024-02-10T08:28:38.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-10T08:28:38.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:35.281002+0000 mon.c (mon.2) 129 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:35.558539+0000 osd.5 (osd.5) 3 : cluster [DBG] 16.3 deep-scrub starts 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:35.611170+0000 osd.5 (osd.5) 4 : cluster [DBG] 16.3 deep-scrub ok 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.193677+0000 mon.c (mon.2) 130 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:36.194139+0000 client.admin (client.?) 0 : cluster [INF] threexx 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.284468+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:36.440672+0000 osd.6 (osd.6) 3 : cluster [DBG] 16.6 deep-scrub starts 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:36.482716+0000 osd.6 (osd.6) 4 : cluster [DBG] 16.6 deep-scrub ok 2024-02-10T08:28:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.592358+0000 mon.a (mon.0) 795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.592567+0000 mon.a (mon.0) 796 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.592721+0000 mon.a (mon.0) 797 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.593782+0000 mon.c (mon.2) 132 : audit [INF] from='client.? 172.21.15.67:0/4191633867' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.594308+0000 mon.b (mon.1) 43 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:36.595836+0000 mon.b (mon.1) 44 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:36.693864+0000 mgr.y (mgr.14547) 241 : cluster [DBG] pgmap v185: 548 pgs: 23 creating+peering, 516 active+clean, 9 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 20 op/s 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.042754+0000 mon.a (mon.0) 798 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.042991+0000 mon.a (mon.0) 799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi067-54245-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:37.043944+0000 mon.a (mon.0) 800 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.057781+0000 mon.a (mon.0) 801 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi067-54261-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.059153+0000 mon.a (mon.0) 802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi067-54503-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.059244+0000 mon.a (mon.0) 803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi067-54285-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.059388+0000 mon.a (mon.0) 804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi067-54509-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: cluster 2024-02-10T08:28:37.059639+0000 mon.a (mon.0) 805 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.062701+0000 mon.a (mon.0) 806 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.063693+0000 mon.a (mon.0) 807 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:38.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.071567+0000 mon.a (mon.0) 808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:38.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.071973+0000 mon.a (mon.0) 809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:38.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.079781+0000 mon.a (mon.0) 810 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:38.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:38 smithi134 bash[18722]: audit 2024-02-10T08:28:37.080521+0000 mon.a (mon.0) 811 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:39.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] api_io: Running main() from gmock_main.cc 2024-02-10T08:28:39.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [==========] Running 23 tests from 2 test suites. 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [----------] Global test environment set-up. 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [----------] 13 tests from LibRadosIo 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (1420 ms) 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: no timeout :/ 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: no timeout :/ 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: no timeout :/ 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: no timeout :/ 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: no timeout :/ 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (463 ms) 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-10T08:28:39.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (9 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.Checksum (5 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (8 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (5 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (6 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.TruncTest (5 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (5 ms) 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-10T08:28:39.530 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.RmXattr (13 ms) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIo.XattrIter (6 ms) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [----------] 13 tests from LibRadosIo (1949 ms total) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1608 ms) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (8 ms) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (6 ms) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (8 ms) 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-10T08:28:39.531 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (11 ms) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (7 ms) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (4 ms) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (6 ms) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (15 ms) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (7 ms) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1680 ms total) 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: 2024-02-10T08:28:39.532 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [----------] Global test environment tear-down 2024-02-10T08:28:39.533 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [==========] 23 tests from 2 test suites ran. (22759 ms total) 2024-02-10T08:28:39.533 INFO:tasks.workunit.client.0.smithi067.stdout: api_io: [ PASSED ] 23 tests. 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: cluster 2024-02-10T08:28:37.086689+0000 osd.1 (osd.1) 3 : cluster [DBG] 16.9 deep-scrub starts 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: cluster 2024-02-10T08:28:37.130396+0000 osd.1 (osd.1) 4 : cluster [DBG] 16.9 deep-scrub ok 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:37.286031+0000 mon.c (mon.2) 133 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.020379+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.023432+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: cluster 2024-02-10T08:28:38.023641+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-10T08:28:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.023795+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.063822+0000 mon.a (mon.0) 814 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.063994+0000 mon.a (mon.0) 815 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.064236+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.287632+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.536718+0000 mon.a (mon.0) 817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: cluster 2024-02-10T08:28:38.536926+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.538654+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.538826+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.67:0/773718600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.540474+0000 mon.a (mon.0) 819 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.548547+0000 mon.a (mon.0) 820 : audit [INF] from='client.? 172.21.15.67:0/2160980737' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi067-54237-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:39.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:39 smithi134 bash[18722]: audit 2024-02-10T08:28:38.564174+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: cluster 2024-02-10T08:28:37.086689+0000 osd.1 (osd.1) 3 : cluster [DBG] 16.9 deep-scrub starts 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: cluster 2024-02-10T08:28:37.130396+0000 osd.1 (osd.1) 4 : cluster [DBG] 16.9 deep-scrub ok 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:37.286031+0000 mon.c (mon.2) 133 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.020379+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.023432+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: cluster 2024-02-10T08:28:38.023641+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-10T08:28:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.023795+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.063822+0000 mon.a (mon.0) 814 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.063994+0000 mon.a (mon.0) 815 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.064236+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.287632+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.536718+0000 mon.a (mon.0) 817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: cluster 2024-02-10T08:28:38.536926+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.538654+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.538826+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.67:0/773718600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.540474+0000 mon.a (mon.0) 819 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.548547+0000 mon.a (mon.0) 820 : audit [INF] from='client.? 172.21.15.67:0/2160980737' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi067-54237-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[17385]: audit 2024-02-10T08:28:38.564174+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: cluster 2024-02-10T08:28:37.086689+0000 osd.1 (osd.1) 3 : cluster [DBG] 16.9 deep-scrub starts 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: cluster 2024-02-10T08:28:37.130396+0000 osd.1 (osd.1) 4 : cluster [DBG] 16.9 deep-scrub ok 2024-02-10T08:28:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:37.286031+0000 mon.c (mon.2) 133 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.020379+0000 mon.a (mon.0) 812 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.023432+0000 mon.c (mon.2) 134 : audit [INF] from='client.? 172.21.15.67:0/1099015088' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: cluster 2024-02-10T08:28:38.023641+0000 client.admin (client.?) 0 : cluster [INF] fourxx 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.023795+0000 mon.a (mon.0) 813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.063822+0000 mon.a (mon.0) 814 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.063994+0000 mon.a (mon.0) 815 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.064236+0000 mon.a (mon.0) 816 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]': finished 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.287632+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.536718+0000 mon.a (mon.0) 817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: cluster 2024-02-10T08:28:38.536926+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.538654+0000 mon.b (mon.1) 45 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.538826+0000 mon.c (mon.2) 136 : audit [INF] from='client.? 172.21.15.67:0/773718600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:39.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.540474+0000 mon.a (mon.0) 819 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:39.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.548547+0000 mon.a (mon.0) 820 : audit [INF] from='client.? 172.21.15.67:0/2160980737' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi067-54237-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:39.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:39 smithi067 bash[24488]: audit 2024-02-10T08:28:38.564174+0000 mon.c (mon.2) 137 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:38.599233+0000 mon.c (mon.2) 138 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:38.694993+0000 mgr.y (mgr.14547) 242 : cluster [DBG] pgmap v188: 580 pgs: 23 creating+peering, 452 active+clean, 105 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 33 MiB/s wr, 40 op/s 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:38.890919+0000 mon.c (mon.2) 139 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:38.892396+0000 mon.c (mon.2) 140 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:38.895441+0000 mgr.y (mgr.14547) 243 : cluster [DBG] pgmap v189: 580 pgs: 23 creating+peering, 452 active+clean, 105 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 36 MiB/s wr, 43 op/s 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.017107+0000 mon.a (mon.0) 821 : audit [DBG] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.026390+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.026576+0000 mon.a (mon.0) 823 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.027362+0000 mon.a (mon.0) 824 : audit [INF] from='client.? 172.21.15.67:0/2160980737' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi067-54237-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:39.139875+0000 osd.0 (osd.0) 5 : cluster [DBG] 16.5 deep-scrub starts 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:39.144562+0000 osd.0 (osd.0) 6 : cluster [DBG] 16.5 deep-scrub ok 2024-02-10T08:28:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.289399+0000 mon.c (mon.2) 141 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:39.510911+0000 osd.4 (osd.4) 3 : cluster [DBG] 16.2 deep-scrub starts 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:39.512858+0000 osd.4 (osd.4) 4 : cluster [DBG] 16.2 deep-scrub ok 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:39.513866+0000 mon.a (mon.0) 825 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.524491+0000 mon.a (mon.0) 826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.525566+0000 mon.a (mon.0) 827 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:38.599233+0000 mon.c (mon.2) 138 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:28:40.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:38.694993+0000 mgr.y (mgr.14547) 242 : cluster [DBG] pgmap v188: 580 pgs: 23 creating+peering, 452 active+clean, 105 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 33 MiB/s wr, 40 op/s 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:38.890919+0000 mon.c (mon.2) 139 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:38.892396+0000 mon.c (mon.2) 140 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:38.895441+0000 mgr.y (mgr.14547) 243 : cluster [DBG] pgmap v189: 580 pgs: 23 creating+peering, 452 active+clean, 105 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 36 MiB/s wr, 43 op/s 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.017107+0000 mon.a (mon.0) 821 : audit [DBG] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.026390+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.026576+0000 mon.a (mon.0) 823 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.027362+0000 mon.a (mon.0) 824 : audit [INF] from='client.? 172.21.15.67:0/2160980737' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi067-54237-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:39.139875+0000 osd.0 (osd.0) 5 : cluster [DBG] 16.5 deep-scrub starts 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:39.144562+0000 osd.0 (osd.0) 6 : cluster [DBG] 16.5 deep-scrub ok 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.289399+0000 mon.c (mon.2) 141 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:39.510911+0000 osd.4 (osd.4) 3 : cluster [DBG] 16.2 deep-scrub starts 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:39.512858+0000 osd.4 (osd.4) 4 : cluster [DBG] 16.2 deep-scrub ok 2024-02-10T08:28:40.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:39.513866+0000 mon.a (mon.0) 825 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.524491+0000 mon.a (mon.0) 826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.525566+0000 mon.a (mon.0) 827 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.525938+0000 mon.a (mon.0) 828 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.526462+0000 mon.a (mon.0) 829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.526790+0000 mon.a (mon.0) 830 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.527016+0000 mon.a (mon.0) 831 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.527168+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.534838+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.538609+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.538907+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.539807+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:39.540528+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:40.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.029112+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.029361+0000 mon.a (mon.0) 837 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.029874+0000 mon.a (mon.0) 838 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.031516+0000 mon.a (mon.0) 839 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.525938+0000 mon.a (mon.0) 828 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.526462+0000 mon.a (mon.0) 829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.526790+0000 mon.a (mon.0) 830 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.527016+0000 mon.a (mon.0) 831 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.527168+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.534838+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.538609+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.538907+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.875 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.539807+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:39.540528+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.029112+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.029361+0000 mon.a (mon.0) 837 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.029874+0000 mon.a (mon.0) 838 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.031516+0000 mon.a (mon.0) 839 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.031830+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.031970+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: cluster 2024-02-10T08:28:40.032050+0000 mon.a (mon.0) 842 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.032835+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:40.876 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.034204+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.043490+0000 mon.a (mon.0) 843 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.044157+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.044172+0000 mon.a (mon.0) 844 : audit [INF] from='client.? 172.21.15.67:0/1566190600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.044807+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.045752+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.046131+0000 mon.a (mon.0) 846 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.047196+0000 mon.a (mon.0) 847 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.047806+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.048286+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[17385]: audit 2024-02-10T08:28:40.048625+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.031830+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.031970+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: cluster 2024-02-10T08:28:40.032050+0000 mon.a (mon.0) 842 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-10T08:28:40.877 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.032835+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.034204+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.043490+0000 mon.a (mon.0) 843 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.044157+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.044172+0000 mon.a (mon.0) 844 : audit [INF] from='client.? 172.21.15.67:0/1566190600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.044807+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.045752+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.046131+0000 mon.a (mon.0) 846 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.047196+0000 mon.a (mon.0) 847 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.047806+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.048286+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:40.878 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:40 smithi067 bash[24488]: audit 2024-02-10T08:28:40.048625+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:38.599233+0000 mon.c (mon.2) 138 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:38.694993+0000 mgr.y (mgr.14547) 242 : cluster [DBG] pgmap v188: 580 pgs: 23 creating+peering, 452 active+clean, 105 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 33 MiB/s wr, 40 op/s 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:38.890919+0000 mon.c (mon.2) 139 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:38.892396+0000 mon.c (mon.2) 140 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:38.895441+0000 mgr.y (mgr.14547) 243 : cluster [DBG] pgmap v189: 580 pgs: 23 creating+peering, 452 active+clean, 105 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 36 MiB/s wr, 43 op/s 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.017107+0000 mon.a (mon.0) 821 : audit [DBG] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-10T08:28:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.026390+0000 mon.a (mon.0) 822 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.026576+0000 mon.a (mon.0) 823 : audit [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi067-54252-15"}]': finished 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.027362+0000 mon.a (mon.0) 824 : audit [INF] from='client.? 172.21.15.67:0/2160980737' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi067-54237-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:39.139875+0000 osd.0 (osd.0) 5 : cluster [DBG] 16.5 deep-scrub starts 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:39.144562+0000 osd.0 (osd.0) 6 : cluster [DBG] 16.5 deep-scrub ok 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.289399+0000 mon.c (mon.2) 141 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:39.510911+0000 osd.4 (osd.4) 3 : cluster [DBG] 16.2 deep-scrub starts 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:39.512858+0000 osd.4 (osd.4) 4 : cluster [DBG] 16.2 deep-scrub ok 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:39.513866+0000 mon.a (mon.0) 825 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.524491+0000 mon.a (mon.0) 826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.525566+0000 mon.a (mon.0) 827 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.525938+0000 mon.a (mon.0) 828 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.526462+0000 mon.a (mon.0) 829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:41.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.526790+0000 mon.a (mon.0) 830 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.527016+0000 mon.a (mon.0) 831 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.527168+0000 mon.c (mon.2) 142 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.534838+0000 mon.a (mon.0) 832 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.538609+0000 mon.a (mon.0) 833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.538907+0000 mon.a (mon.0) 834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.539807+0000 mon.c (mon.2) 143 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:39.540528+0000 mon.a (mon.0) 835 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.029112+0000 mon.a (mon.0) 836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.029361+0000 mon.a (mon.0) 837 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.029874+0000 mon.a (mon.0) 838 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:41.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.031516+0000 mon.a (mon.0) 839 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.031830+0000 mon.a (mon.0) 840 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-4", "overlaypool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.031970+0000 mon.a (mon.0) 841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: cluster 2024-02-10T08:28:40.032050+0000 mon.a (mon.0) 842 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.032835+0000 mon.c (mon.2) 144 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.034204+0000 mon.c (mon.2) 145 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.043490+0000 mon.a (mon.0) 843 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.044157+0000 mon.b (mon.1) 46 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.044172+0000 mon.a (mon.0) 844 : audit [INF] from='client.? 172.21.15.67:0/1566190600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.044807+0000 mon.b (mon.1) 47 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.045752+0000 mon.a (mon.0) 845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.046131+0000 mon.a (mon.0) 846 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.047196+0000 mon.a (mon.0) 847 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:41.030 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.047806+0000 mon.a (mon.0) 848 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:28:41.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.048286+0000 mon.a (mon.0) 849 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:41.031 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:40 smithi134 bash[18722]: audit 2024-02-10T08:28:40.048625+0000 mon.a (mon.0) 850 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout:LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (6 ms) 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (7 ms) 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (6 ms) 2024-02-10T08:28:41.516 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (4 ms) 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (15 ms) 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (5 ms) 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-10T08:28:41.517 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (9 ms) 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (7 ms) 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1202 ms total) 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (24744 ms total) 2024-02-10T08:28:41.518 INFO:tasks.workunit.client.0.smithi067.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout:adosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.062701+0000 mon.a [INF] from='client.? 172.21.15.67:0/1278324698' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi067-54252-15"}]: dispatch 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.063693+0000 mon.a [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.071567+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.071973+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi067-54538-1"}]: dispatch 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.079781+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi067-54293-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: got: 2024-02-10T08:28:37.080521+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (9855 ms) 2024-02-10T08:28:41.528 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (24629 ms total) 2024-02-10T08:28:41.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: 2024-02-10T08:28:41.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-10T08:28:41.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (24629 ms total) 2024-02-10T08:28:41.529 INFO:tasks.workunit.client.0.smithi067.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-10T08:28:42.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:28:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:28:42.468 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-10T08:28:42.468 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: seed 54509 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (265 ms) 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (16 ms) 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (315 ms) 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (502 ms) 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (62 ms) 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (1160 ms total) 2024-02-10T08:28:42.469 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1104 ms) 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (4 ms) 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (10 ms) 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1118 ms total) 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-10T08:28:42.470 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (25602 ms total) 2024-02-10T08:28:42.471 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: Running main() from gmock_main.cc 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [----------] Global test environment set-up. 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStat.Stat (277 ms) 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStat.StatNS (32 ms) 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-10T08:28:42.478 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (293 ms) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (502 ms) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [----------] 4 tests from LibRadosStat (1104 ms total) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1093 ms) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (10 ms) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (4 ms) 2024-02-10T08:28:42.479 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1108 ms total) 2024-02-10T08:28:42.480 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: 2024-02-10T08:28:42.480 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [----------] Global test environment tear-down 2024-02-10T08:28:42.480 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (25616 ms total) 2024-02-10T08:28:42.480 INFO:tasks.workunit.client.0.smithi067.stdout: api_stat: [ PASSED ] 8 tests. 2024-02-10T08:28:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:40.093057+0000 mon.a (mon.0) 851 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pg_num": 4}]: dispatch 2024-02-10T08:28:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:40.291114+0000 mon.c (mon.2) 146 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: cluster 2024-02-10T08:28:40.551937+0000 mon.a (mon.0) 852 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-10T08:28:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: cluster 2024-02-10T08:28:41.023887+0000 mon.a (mon.0) 853 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.036237+0000 mon.a (mon.0) 854 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.036599+0000 mon.a (mon.0) 855 : audit [INF] from='client.? 172.21.15.67:0/1566190600' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.036713+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.040818+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.292828+0000 mon.c (mon.2) 148 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.504144+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.504438+0000 mon.a (mon.0) 858 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.504663+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.505424+0000 mon.a (mon.0) 860 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pg_num": 4}]': finished 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: cluster 2024-02-10T08:28:41.505694+0000 mon.a (mon.0) 861 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.505994+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.507598+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:42.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.508851+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.509783+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.512004+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.512967+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.514357+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.516051+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.523777+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]: dispatch 2024-02-10T08:28:42.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:42 smithi134 bash[18722]: audit 2024-02-10T08:28:41.523906+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]: dispatch 2024-02-10T08:28:42.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:40.093057+0000 mon.a (mon.0) 851 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pg_num": 4}]: dispatch 2024-02-10T08:28:42.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:40.291114+0000 mon.c (mon.2) 146 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:42.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: cluster 2024-02-10T08:28:40.551937+0000 mon.a (mon.0) 852 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: cluster 2024-02-10T08:28:41.023887+0000 mon.a (mon.0) 853 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.036237+0000 mon.a (mon.0) 854 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.036599+0000 mon.a (mon.0) 855 : audit [INF] from='client.? 172.21.15.67:0/1566190600' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.036713+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.040818+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.292828+0000 mon.c (mon.2) 148 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.504144+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.504438+0000 mon.a (mon.0) 858 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.504663+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.505424+0000 mon.a (mon.0) 860 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pg_num": 4}]': finished 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: cluster 2024-02-10T08:28:41.505694+0000 mon.a (mon.0) 861 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.505994+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.507598+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.508851+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.509783+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.512004+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.512967+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.514357+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.516051+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.523777+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[24488]: audit 2024-02-10T08:28:41.523906+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]: dispatch 2024-02-10T08:28:42.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:40.093057+0000 mon.a (mon.0) 851 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pg_num": 4}]: dispatch 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:40.291114+0000 mon.c (mon.2) 146 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: cluster 2024-02-10T08:28:40.551937+0000 mon.a (mon.0) 852 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: cluster 2024-02-10T08:28:41.023887+0000 mon.a (mon.0) 853 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.036237+0000 mon.a (mon.0) 854 : audit [INF] from='client.? 172.21.15.67:0/996301038' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi067-54261-22"}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.036599+0000 mon.a (mon.0) 855 : audit [INF] from='client.? 172.21.15.67:0/1566190600' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.036713+0000 mon.a (mon.0) 856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.040818+0000 mon.c (mon.2) 147 : audit [INF] from='client.? 172.21.15.67:0/1056496548' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.292828+0000 mon.c (mon.2) 148 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.504144+0000 mon.a (mon.0) 857 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-4-cache", "mode": "writeback"}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.504438+0000 mon.a (mon.0) 858 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi067-54299-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.504663+0000 mon.a (mon.0) 859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.505424+0000 mon.a (mon.0) 860 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pg_num": 4}]': finished 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: cluster 2024-02-10T08:28:41.505694+0000 mon.a (mon.0) 861 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-10T08:28:42.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.505994+0000 mon.b (mon.1) 48 : audit [INF] from='client.? 172.21.15.67:0/159569819' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.507598+0000 mon.c (mon.2) 149 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.508851+0000 mon.a (mon.0) 862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.509783+0000 mon.a (mon.0) 863 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.512004+0000 mon.a (mon.0) 864 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.512967+0000 mon.a (mon.0) 865 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.514357+0000 mon.a (mon.0) 866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.516051+0000 mon.a (mon.0) 867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.523777+0000 mon.b (mon.1) 49 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]: dispatch 2024-02-10T08:28:42.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:42 smithi067 bash[17385]: audit 2024-02-10T08:28:41.523906+0000 mon.a (mon.0) 868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]: dispatch 2024-02-10T08:28:43.454 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:28:43.484 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: Running main() from gmock_main.cc 2024-02-10T08:28:43.484 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-10T08:28:43.484 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [----------] Global test environment set-up. 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (1311 ms) 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.LockShared (7 ms) 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1008 ms) 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1046 ms) 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (5 ms) 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.Unlock (5 ms) 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-10T08:28:43.485 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (7 ms) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (4 ms) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [----------] 8 tests from LibRadosLock (3393 ms total) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1049 ms) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (5 ms) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1054 ms) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1008 ms) 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-10T08:28:43.486 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (8 ms) 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (7 ms) 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (8 ms) 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (6 ms) 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3145 ms total) 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [----------] Global test environment tear-down 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (26699 ms total) 2024-02-10T08:28:43.487 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-10T08:28:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: cluster 2024-02-10T08:28:40.896257+0000 mgr.y (mgr.14547) 244 : cluster [DBG] pgmap v192: 628 pgs: 23 creating+peering, 452 active+clean, 153 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:28:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:41.853671+0000 mgr.y (mgr.14547) 245 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: cluster 2024-02-10T08:28:42.056594+0000 osd.3 (osd.3) 3 : cluster [DBG] 16.4 deep-scrub starts 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: cluster 2024-02-10T08:28:42.059702+0000 osd.3 (osd.3) 4 : cluster [DBG] 16.4 deep-scrub ok 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: cluster 2024-02-10T08:28:42.109639+0000 osd.0 (osd.0) 7 : cluster [DBG] 16.8 deep-scrub starts 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: cluster 2024-02-10T08:28:42.112994+0000 osd.0 (osd.0) 8 : cluster [DBG] 16.8 deep-scrub ok 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.294290+0000 mon.c (mon.2) 150 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.461831+0000 mon.a (mon.0) 869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.461972+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.462055+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.462215+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.462311+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.462397+0000 mon.a (mon.0) 874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:43.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.462490+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]': finished 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: cluster 2024-02-10T08:28:42.462715+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.464104+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.465278+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.67:0/4145563726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.465855+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.67:0/310960088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.467629+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.467934+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.469769+0000 mon.a (mon.0) 879 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.470156+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.470347+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.470626+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:42.473556+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:43.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:43 smithi134 bash[18722]: audit 2024-02-10T08:28:43.295881+0000 mon.c (mon.2) 154 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: cluster 2024-02-10T08:28:40.896257+0000 mgr.y (mgr.14547) 244 : cluster [DBG] pgmap v192: 628 pgs: 23 creating+peering, 452 active+clean, 153 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:41.853671+0000 mgr.y (mgr.14547) 245 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: cluster 2024-02-10T08:28:42.056594+0000 osd.3 (osd.3) 3 : cluster [DBG] 16.4 deep-scrub starts 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: cluster 2024-02-10T08:28:42.059702+0000 osd.3 (osd.3) 4 : cluster [DBG] 16.4 deep-scrub ok 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: cluster 2024-02-10T08:28:42.109639+0000 osd.0 (osd.0) 7 : cluster [DBG] 16.8 deep-scrub starts 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: cluster 2024-02-10T08:28:42.112994+0000 osd.0 (osd.0) 8 : cluster [DBG] 16.8 deep-scrub ok 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.294290+0000 mon.c (mon.2) 150 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.461831+0000 mon.a (mon.0) 869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.461972+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.462055+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.462215+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.462311+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.462397+0000 mon.a (mon.0) 874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.462490+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]': finished 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: cluster 2024-02-10T08:28:42.462715+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.464104+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.465278+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.67:0/4145563726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: cluster 2024-02-10T08:28:40.896257+0000 mgr.y (mgr.14547) 244 : cluster [DBG] pgmap v192: 628 pgs: 23 creating+peering, 452 active+clean, 153 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:41.853671+0000 mgr.y (mgr.14547) 245 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: cluster 2024-02-10T08:28:42.056594+0000 osd.3 (osd.3) 3 : cluster [DBG] 16.4 deep-scrub starts 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: cluster 2024-02-10T08:28:42.059702+0000 osd.3 (osd.3) 4 : cluster [DBG] 16.4 deep-scrub ok 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: cluster 2024-02-10T08:28:42.109639+0000 osd.0 (osd.0) 7 : cluster [DBG] 16.8 deep-scrub starts 2024-02-10T08:28:43.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: cluster 2024-02-10T08:28:42.112994+0000 osd.0 (osd.0) 8 : cluster [DBG] 16.8 deep-scrub ok 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.294290+0000 mon.c (mon.2) 150 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.461831+0000 mon.a (mon.0) 869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi067-54526-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.461972+0000 mon.a (mon.0) 870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi067-54509-7"}]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.462055+0000 mon.a (mon.0) 871 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.462215+0000 mon.a (mon.0) 872 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.462311+0000 mon.a (mon.0) 873 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.462397+0000 mon.a (mon.0) 874 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi067-54503-6"}]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.462490+0000 mon.a (mon.0) 875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-4"}]': finished 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: cluster 2024-02-10T08:28:42.462715+0000 mon.a (mon.0) 876 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.464104+0000 mon.c (mon.2) 151 : audit [INF] from='client.? 172.21.15.67:0/2020712584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:43.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.465278+0000 mon.c (mon.2) 152 : audit [INF] from='client.? 172.21.15.67:0/4145563726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.465855+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.67:0/310960088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.467629+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.467934+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.469769+0000 mon.a (mon.0) 879 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.470156+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.470347+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.470626+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:42.473556+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[24488]: audit 2024-02-10T08:28:43.295881+0000 mon.c (mon.2) 154 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:43.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.465855+0000 mon.c (mon.2) 153 : audit [INF] from='client.? 172.21.15.67:0/310960088' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.467629+0000 mon.a (mon.0) 877 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.467934+0000 mon.a (mon.0) 878 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.469769+0000 mon.a (mon.0) 879 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.470156+0000 mon.b (mon.1) 50 : audit [INF] from='client.? 172.21.15.67:0/1249069961' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.470347+0000 mon.a (mon.0) 880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.470626+0000 mon.a (mon.0) 881 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:42.473556+0000 mon.a (mon.0) 882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]: dispatch 2024-02-10T08:28:43.875 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:43 smithi067 bash[17385]: audit 2024-02-10T08:28:43.295881+0000 mon.c (mon.2) 154 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: seed 54293 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (6 ms) 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (4 ms) 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-10T08:28:45.220 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1008 ms) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1006 ms) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (6 ms) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (7 ms) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (4 ms) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2046 ms total) 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-10T08:28:45.221 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1051 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (5 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1008 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1007 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (7 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (6 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (7 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (4 ms) 2024-02-10T08:28:45.222 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3095 ms total) 2024-02-10T08:28:45.223 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: 2024-02-10T08:28:45.223 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-10T08:28:45.223 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (28428 ms total) 2024-02-10T08:28:45.223 INFO:tasks.workunit.client.0.smithi067.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-10T08:28:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:42.897397+0000 mgr.y (mgr.14547) 246 : cluster [DBG] pgmap v195: 600 pgs: 4 creating+peering, 524 active+clean, 72 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-02-10T08:28:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:43.464006+0000 mon.a (mon.0) 883 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:28:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:43.464469+0000 mon.a (mon.0) 884 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.474686+0000 mon.a (mon.0) 885 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]': finished 2024-02-10T08:28:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.474858+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.475026+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.475155+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.475302+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.475468+0000 mon.a (mon.0) 890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:43.475927+0000 mon.a (mon.0) 891 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.479564+0000 mon.a (mon.0) 892 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "mode": "writeback"}]: dispatch 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.482371+0000 mon.a (mon.0) 893 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.485750+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.67:0/3471712665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.485808+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.495415+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:43.495756+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:44.016979+0000 mon.a (mon.0) 896 : cluster [WRN] pool 'PoolQuotaPP_smithi067-54245-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:44.018654+0000 mon.a (mon.0) 897 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:44.018712+0000 mon.a (mon.0) 898 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.027578+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "mode": "writeback"}]': finished 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.027753+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.027848+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.028045+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: cluster 2024-02-10T08:28:44.028361+0000 mon.a (mon.0) 903 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.040520+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.040531+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.048505+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.67:0/1608248387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.051861+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.052177+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:45 smithi134 bash[18722]: audit 2024-02-10T08:28:44.057705+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:42.897397+0000 mgr.y (mgr.14547) 246 : cluster [DBG] pgmap v195: 600 pgs: 4 creating+peering, 524 active+clean, 72 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:43.464006+0000 mon.a (mon.0) 883 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:43.464469+0000 mon.a (mon.0) 884 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.474686+0000 mon.a (mon.0) 885 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]': finished 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.474858+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.475026+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.475155+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.475302+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.475468+0000 mon.a (mon.0) 890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:43.475927+0000 mon.a (mon.0) 891 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-10T08:28:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.479564+0000 mon.a (mon.0) 892 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "mode": "writeback"}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.482371+0000 mon.a (mon.0) 893 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.485750+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.67:0/3471712665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.485808+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.495415+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:43.495756+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:44.016979+0000 mon.a (mon.0) 896 : cluster [WRN] pool 'PoolQuotaPP_smithi067-54245-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:44.018654+0000 mon.a (mon.0) 897 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:44.018712+0000 mon.a (mon.0) 898 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.027578+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "mode": "writeback"}]': finished 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.027753+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.027848+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.028045+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: cluster 2024-02-10T08:28:44.028361+0000 mon.a (mon.0) 903 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.040520+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.040531+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.048505+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.67:0/1608248387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.051861+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.052177+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[24488]: audit 2024-02-10T08:28:44.057705+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:42.897397+0000 mgr.y (mgr.14547) 246 : cluster [DBG] pgmap v195: 600 pgs: 4 creating+peering, 524 active+clean, 72 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 8.7 KiB/s wr, 10 op/s 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:43.464006+0000 mon.a (mon.0) 883 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:43.464469+0000 mon.a (mon.0) 884 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.474686+0000 mon.a (mon.0) 885 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]': finished 2024-02-10T08:28:45.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.474858+0000 mon.a (mon.0) 886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi067-54285-10"}]': finished 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.475026+0000 mon.a (mon.0) 887 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.475155+0000 mon.a (mon.0) 888 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.475302+0000 mon.a (mon.0) 889 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi067-54237-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.475468+0000 mon.a (mon.0) 890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-4", "tierpool": "test-rados-api-smithi067-54339-4-cache"}]': finished 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:43.475927+0000 mon.a (mon.0) 891 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.479564+0000 mon.a (mon.0) 892 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "mode": "writeback"}]: dispatch 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.482371+0000 mon.a (mon.0) 893 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.485750+0000 mon.c (mon.2) 155 : audit [INF] from='client.? 172.21.15.67:0/3471712665' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.485808+0000 mon.b (mon.1) 51 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.495415+0000 mon.a (mon.0) 894 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:43.495756+0000 mon.a (mon.0) 895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:44.016979+0000 mon.a (mon.0) 896 : cluster [WRN] pool 'PoolQuotaPP_smithi067-54245-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:44.018654+0000 mon.a (mon.0) 897 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:28:45.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:44.018712+0000 mon.a (mon.0) 898 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.027578+0000 mon.a (mon.0) 899 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "mode": "writeback"}]': finished 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.027753+0000 mon.a (mon.0) 900 : audit [INF] from='client.? 172.21.15.67:0/3933013755' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi067-54299-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.027848+0000 mon.a (mon.0) 901 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.028045+0000 mon.a (mon.0) 902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: cluster 2024-02-10T08:28:44.028361+0000 mon.a (mon.0) 903 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.040520+0000 mon.b (mon.1) 52 : audit [INF] from='client.? 172.21.15.67:0/205529988' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.040531+0000 mon.c (mon.2) 156 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.048505+0000 mon.c (mon.2) 157 : audit [INF] from='client.? 172.21.15.67:0/1608248387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.051861+0000 mon.a (mon.0) 904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.052177+0000 mon.a (mon.0) 905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:45.625 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:45 smithi067 bash[17385]: audit 2024-02-10T08:28:44.057705+0000 mon.a (mon.0) 906 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]: dispatch 2024-02-10T08:28:46.435 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify_test_cb 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (13 ms) 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94376930635648 err -107 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-10T08:28:46.436 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (10 ms) 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94376930635648 err -107waiting up to 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: 300 for disconnect notification ... 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (8 ms) 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 292057776128 cookie 94376930651728 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (14 ms) 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 292057776129 cookie 94376930652992 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (8 ms) 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 292057776130 cookie 94376930652992 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (7 ms) 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-10T08:28:46.437 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 292057776129 cookie 94376930652992 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 292057776129 cookie 94376930683120 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (12 ms) 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 292057776129 cookie 94376930683120 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 296352743426 cookie 94376930683120 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3008 ms) 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94376930683120 err -107 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_cb from 24709 notify_id 322122547204 cookie 94376930652992 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5015 ms) 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94376930652992 err -107 2024-02-10T08:28:46.438 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1009 ms) 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9104 ms total) 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: watch_notify_test_cb 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1088 ms) 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1088 ms total) 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (29558 ms total) 2024-02-10T08:28:46.439 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:44.297641+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: cluster 2024-02-10T08:28:44.898152+0000 mgr.y (mgr.14547) 247 : cluster [DBG] pgmap v198: 488 pgs: 4 creating+peering, 420 active+clean, 64 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 2 op/s 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: cluster 2024-02-10T08:28:45.038559+0000 osd.3 (osd.3) 5 : cluster [DBG] 16.7 deep-scrub starts 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: cluster 2024-02-10T08:28:45.041978+0000 osd.3 (osd.3) 6 : cluster [DBG] 16.7 deep-scrub ok 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.174236+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.174355+0000 mon.a (mon.0) 908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.174466+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: cluster 2024-02-10T08:28:45.174800+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.188219+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.67:0/3815657935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.188425+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.67:0/3830171879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi067-54237-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.199772+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.299671+0000 mon.c (mon.2) 160 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.755763+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.756166+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[24488]: audit 2024-02-10T08:28:45.774074+0000 mon.a (mon.0) 914 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:44.297641+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: cluster 2024-02-10T08:28:44.898152+0000 mgr.y (mgr.14547) 247 : cluster [DBG] pgmap v198: 488 pgs: 4 creating+peering, 420 active+clean, 64 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 2 op/s 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: cluster 2024-02-10T08:28:45.038559+0000 osd.3 (osd.3) 5 : cluster [DBG] 16.7 deep-scrub starts 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: cluster 2024-02-10T08:28:45.041978+0000 osd.3 (osd.3) 6 : cluster [DBG] 16.7 deep-scrub ok 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.174236+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:46.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.174355+0000 mon.a (mon.0) 908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.174466+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: cluster 2024-02-10T08:28:45.174800+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.188219+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.67:0/3815657935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.188425+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.67:0/3830171879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi067-54237-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.199772+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.299671+0000 mon.c (mon.2) 160 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.755763+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.756166+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:46.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:46 smithi067 bash[17385]: audit 2024-02-10T08:28:45.774074+0000 mon.a (mon.0) 914 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:44.297641+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: cluster 2024-02-10T08:28:44.898152+0000 mgr.y (mgr.14547) 247 : cluster [DBG] pgmap v198: 488 pgs: 4 creating+peering, 420 active+clean, 64 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 2 op/s 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: cluster 2024-02-10T08:28:45.038559+0000 osd.3 (osd.3) 5 : cluster [DBG] 16.7 deep-scrub starts 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: cluster 2024-02-10T08:28:45.041978+0000 osd.3 (osd.3) 6 : cluster [DBG] 16.7 deep-scrub ok 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.174236+0000 mon.a (mon.0) 907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.174355+0000 mon.a (mon.0) 908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi067-54461-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.174466+0000 mon.a (mon.0) 909 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi067-54293-10"}]': finished 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: cluster 2024-02-10T08:28:45.174800+0000 mon.a (mon.0) 910 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.188219+0000 mon.b (mon.1) 53 : audit [INF] from='client.? 172.21.15.67:0/3815657935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.188425+0000 mon.a (mon.0) 911 : audit [INF] from='client.? 172.21.15.67:0/3830171879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi067-54237-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.199772+0000 mon.c (mon.2) 159 : audit [INF] from='client.? 172.21.15.67:0/1059801637' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:46.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.299671+0000 mon.c (mon.2) 160 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:46.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.755763+0000 mon.a (mon.0) 912 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:46.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.756166+0000 mon.a (mon.0) 913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]: dispatch 2024-02-10T08:28:46.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:46 smithi134 bash[18722]: audit 2024-02-10T08:28:45.774074+0000 mon.a (mon.0) 914 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:28:47.300 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3173 ms) 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (8 ms) 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (3181 ms total) 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify 2024-02-10T08:28:47.301 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (48 ms) 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4056 ms) 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (7 ms) 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (23 ms) 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422096688 notify_id 356482285569 notifier_gid 15222 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (15 ms) 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422096688 notify_id 356482285570 notifier_gid 15222 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (9 ms) 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422096688 notify_id 356482285571 notifier_gid 15222 2024-02-10T08:28:47.302 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (27 ms) 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422096688 notify_id 356482285569 notifier_gid 15222 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (19 ms) 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422137360 notify_id 356482285570 notifier_gid 15222 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (13 ms) 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422137360 notify_id 356482285570 notifier_gid 15222 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (18 ms) 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: trying... 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422137360 notify_id 356482285570 notifier_gid 15222 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: timed out 2024-02-10T08:28:47.303 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: flushing 2024-02-10T08:28:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.303224+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.378871+0000 mon.a (mon.0) 915 : audit [INF] from='client.? 172.21.15.67:0/3830171879' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi067-54237-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.379016+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.379094+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: cluster 2024-02-10T08:28:46.379323+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.401816+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.67:0/3830020297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.419634+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.427629+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.67:0/3315015422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:46.476438+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17385]: audit 2024-02-10T08:28:47.305772+0000 mon.c (mon.2) 164 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.303224+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.378871+0000 mon.a (mon.0) 915 : audit [INF] from='client.? 172.21.15.67:0/3830171879' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi067-54237-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.379016+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.379094+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: cluster 2024-02-10T08:28:46.379323+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.401816+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.67:0/3830020297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.419634+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.427629+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.67:0/3315015422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:46.476438+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[24488]: audit 2024-02-10T08:28:47.305772+0000 mon.c (mon.2) 164 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.303224+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.378871+0000 mon.a (mon.0) 915 : audit [INF] from='client.? 172.21.15.67:0/3830171879' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi067-54237-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.379016+0000 mon.a (mon.0) 916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi067-54485-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.379094+0000 mon.a (mon.0) 917 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi067-54526-12"}]': finished 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: cluster 2024-02-10T08:28:46.379323+0000 mon.a (mon.0) 918 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.401816+0000 mon.c (mon.2) 162 : audit [INF] from='client.? 172.21.15.67:0/3830020297' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.419634+0000 mon.a (mon.0) 919 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.427629+0000 mon.c (mon.2) 163 : audit [INF] from='client.? 172.21.15.67:0/3315015422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:46.476438+0000 mon.a (mon.0) 920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:47 smithi134 bash[18722]: audit 2024-02-10T08:28:47.305772+0000 mon.c (mon.2) 164 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:28:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:47] "GET /metrics HTTP/1.1" 200 204699 "" "Prometheus/2.33.4" 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5058 ms) 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-10T08:28:48.399 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (35 ms) 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (21245 ms) 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: attempt 0 of 20 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5117 ms) 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (31455 ms total) 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (31455 ms total) 2024-02-10T08:28:48.400 INFO:tasks.workunit.client.0.smithi067.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-10T08:28:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: cluster 2024-02-10T08:28:46.901379+0000 mgr.y (mgr.14547) 248 : cluster [DBG] pgmap v201: 520 pgs: 64 creating+peering, 456 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: audit 2024-02-10T08:28:47.377002+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: audit 2024-02-10T08:28:47.377138+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: cluster 2024-02-10T08:28:47.377244+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: audit 2024-02-10T08:28:47.398987+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: audit 2024-02-10T08:28:47.404213+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: audit 2024-02-10T08:28:48.308701+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:48 smithi134 bash[18722]: audit 2024-02-10T08:28:48.371692+0000 mon.a (mon.0) 925 : audit [DBG] from='client.? 172.21.15.67:0/104532712' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: cluster 2024-02-10T08:28:46.901379+0000 mgr.y (mgr.14547) 248 : cluster [DBG] pgmap v201: 520 pgs: 64 creating+peering, 456 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: audit 2024-02-10T08:28:47.377002+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: audit 2024-02-10T08:28:47.377138+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: cluster 2024-02-10T08:28:47.377244+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: audit 2024-02-10T08:28:47.398987+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: audit 2024-02-10T08:28:47.404213+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: audit 2024-02-10T08:28:48.308701+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[17385]: audit 2024-02-10T08:28:48.371692+0000 mon.a (mon.0) 925 : audit [DBG] from='client.? 172.21.15.67:0/104532712' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: cluster 2024-02-10T08:28:46.901379+0000 mgr.y (mgr.14547) 248 : cluster [DBG] pgmap v201: 520 pgs: 64 creating+peering, 456 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: audit 2024-02-10T08:28:47.377002+0000 mon.a (mon.0) 921 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: audit 2024-02-10T08:28:47.377138+0000 mon.a (mon.0) 922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: cluster 2024-02-10T08:28:47.377244+0000 mon.a (mon.0) 923 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: audit 2024-02-10T08:28:47.398987+0000 mon.c (mon.2) 165 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: audit 2024-02-10T08:28:47.404213+0000 mon.a (mon.0) 924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: audit 2024-02-10T08:28:48.308701+0000 mon.c (mon.2) 166 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:48 smithi067 bash[24488]: audit 2024-02-10T08:28:48.371692+0000 mon.a (mon.0) 925 : audit [DBG] from='client.? 172.21.15.67:0/104532712' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T08:28:49.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[24488]: audit 2024-02-10T08:28:48.372010+0000 mgr.y (mgr.14547) 249 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T08:28:49.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[24488]: audit 2024-02-10T08:28:48.407750+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:49.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[24488]: cluster 2024-02-10T08:28:48.408002+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-10T08:28:49.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[24488]: audit 2024-02-10T08:28:48.419146+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.67:0/3210622677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:49.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[24488]: audit 2024-02-10T08:28:48.421224+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:49.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[17385]: audit 2024-02-10T08:28:48.372010+0000 mgr.y (mgr.14547) 249 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T08:28:49.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[17385]: audit 2024-02-10T08:28:48.407750+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:49.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[17385]: cluster 2024-02-10T08:28:48.408002+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-10T08:28:49.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[17385]: audit 2024-02-10T08:28:48.419146+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.67:0/3210622677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:49.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:49 smithi067 bash[17385]: audit 2024-02-10T08:28:48.421224+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:49 smithi134 bash[18722]: audit 2024-02-10T08:28:48.372010+0000 mgr.y (mgr.14547) 249 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-10T08:28:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:49 smithi134 bash[18722]: audit 2024-02-10T08:28:48.407750+0000 mon.a (mon.0) 926 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:49 smithi134 bash[18722]: cluster 2024-02-10T08:28:48.408002+0000 mon.a (mon.0) 927 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-10T08:28:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:49 smithi134 bash[18722]: audit 2024-02-10T08:28:48.419146+0000 mon.c (mon.2) 167 : audit [INF] from='client.? 172.21.15.67:0/3210622677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:49 smithi134 bash[18722]: audit 2024-02-10T08:28:48.421224+0000 mon.a (mon.0) 928 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: cluster 2024-02-10T08:28:48.902102+0000 mgr.y (mgr.14547) 250 : cluster [DBG] pgmap v204: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:49.310145+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: cluster 2024-02-10T08:28:49.550421+0000 mon.a (mon.0) 929 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:49.583587+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:49.606436+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.67:0/719051612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:49.610923+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: cluster 2024-02-10T08:28:50.149791+0000 mon.a (mon.0) 931 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:50.160093+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:50.172499+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[17385]: audit 2024-02-10T08:28:50.311786+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: cluster 2024-02-10T08:28:48.902102+0000 mgr.y (mgr.14547) 250 : cluster [DBG] pgmap v204: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:49.310145+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:50.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: cluster 2024-02-10T08:28:49.550421+0000 mon.a (mon.0) 929 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:49.583587+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:49.606436+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.67:0/719051612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:49.610923+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: cluster 2024-02-10T08:28:50.149791+0000 mon.a (mon.0) 931 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:50.160093+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:50.172499+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:50.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:50 smithi067 bash[24488]: audit 2024-02-10T08:28:50.311786+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: cluster 2024-02-10T08:28:48.902102+0000 mgr.y (mgr.14547) 250 : cluster [DBG] pgmap v204: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:49.310145+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: cluster 2024-02-10T08:28:49.550421+0000 mon.a (mon.0) 929 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:49.583587+0000 mon.a (mon.0) 930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi067-54237-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:49.606436+0000 mon.b (mon.1) 54 : audit [INF] from='client.? 172.21.15.67:0/719051612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:49.610923+0000 mon.c (mon.2) 169 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: cluster 2024-02-10T08:28:50.149791+0000 mon.a (mon.0) 931 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-10T08:28:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:50.160093+0000 mon.a (mon.0) 932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:50.172499+0000 mon.a (mon.0) 933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:50 smithi134 bash[18722]: audit 2024-02-10T08:28:50.311786+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:51.866 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: cluster 2024-02-10T08:28:50.573148+0000 mon.a (mon.0) 934 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:28:51.866 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:50.610054+0000 mon.a (mon.0) 935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:50.610223+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: cluster 2024-02-10T08:28:50.610311+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:50.630423+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.67:0/939600426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:50.631634+0000 mon.a (mon.0) 938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:50.742110+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:50.743737+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:51.867 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[18722]: audit 2024-02-10T08:28:51.312958+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:51.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: cluster 2024-02-10T08:28:50.573148+0000 mon.a (mon.0) 934 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:28:51.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:50.610054+0000 mon.a (mon.0) 935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:50.610223+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: cluster 2024-02-10T08:28:50.610311+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:50.630423+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.67:0/939600426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:50.631634+0000 mon.a (mon.0) 938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:50.742110+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:50.743737+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[17385]: audit 2024-02-10T08:28:51.312958+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: cluster 2024-02-10T08:28:50.573148+0000 mon.a (mon.0) 934 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:28:51.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:50.610054+0000 mon.a (mon.0) 935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:50.610223+0000 mon.a (mon.0) 936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: cluster 2024-02-10T08:28:50.610311+0000 mon.a (mon.0) 937 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:50.630423+0000 mon.b (mon.1) 55 : audit [INF] from='client.? 172.21.15.67:0/939600426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:50.631634+0000 mon.a (mon.0) 938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:50.742110+0000 mon.c (mon.2) 171 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:50.743737+0000 mon.a (mon.0) 939 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:51.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:51 smithi067 bash[24488]: audit 2024-02-10T08:28:51.312958+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:52.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:28:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:28:53.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: cluster 2024-02-10T08:28:50.902873+0000 mgr.y (mgr.14547) 251 : cluster [DBG] pgmap v207: 520 pgs: 128 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:51.591445+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:51.591610+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: cluster 2024-02-10T08:28:51.591697+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:51.602862+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.67:0/3336940879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:51.619347+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:51.621916+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:51.865040+0000 mgr.y (mgr.14547) 252 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:52.194587+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[17385]: audit 2024-02-10T08:28:52.318623+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: cluster 2024-02-10T08:28:50.902873+0000 mgr.y (mgr.14547) 251 : cluster [DBG] pgmap v207: 520 pgs: 128 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:28:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:51.591445+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:51.591610+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: cluster 2024-02-10T08:28:51.591697+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:51.602862+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.67:0/3336940879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:51.619347+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:51.621916+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:51.865040+0000 mgr.y (mgr.14547) 252 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:52.194587+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:53.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:52 smithi067 bash[24488]: audit 2024-02-10T08:28:52.318623+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: cluster 2024-02-10T08:28:50.902873+0000 mgr.y (mgr.14547) 251 : cluster [DBG] pgmap v207: 520 pgs: 128 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:28:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:51.591445+0000 mon.a (mon.0) 940 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:51.591610+0000 mon.a (mon.0) 941 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:28:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: cluster 2024-02-10T08:28:51.591697+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-10T08:28:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:51.602862+0000 mon.c (mon.2) 173 : audit [INF] from='client.? 172.21.15.67:0/3336940879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:51.619347+0000 mon.a (mon.0) 943 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:53.150 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:51.621916+0000 mon.c (mon.2) 174 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:53.150 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:51.865040+0000 mgr.y (mgr.14547) 252 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:28:53.150 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:52.194587+0000 mon.a (mon.0) 944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:53.150 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:52 smithi134 bash[18722]: audit 2024-02-10T08:28:52.318623+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: io: 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: client: 1.2 KiB/s rd, 127 B/s wr, 1 op/s rd, 0 op/s wr 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: cluster: 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: id: 2dd26d0c-c7ec-11ee-95b8-87774f69a715 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: health: HEALTH_OK 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: services: 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-02-10T08:28:53.248 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: data: 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pools: 26 pools, 748 pgs 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: objects: 192 objects, 5.7 KiB 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: pgs: 31.684% pgs unknown 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 50.668% pgs not active 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 379 creating+peering 2024-02-10T08:28:53.249 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 237 unknown 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 132 active+clean 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ OK ] LibRadosService.StatusFormat (10303 ms) 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ OK ] LibRadosService.Status (20937 ms) 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [----------] 4 tests from LibRadosService (36305 ms total) 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [----------] Global test environment tear-down 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [==========] 4 tests from 1 test suite ran. (36305 ms total) 2024-02-10T08:28:53.250 INFO:tasks.workunit.client.0.smithi067.stdout: api_service: [ PASSED ] 4 tests. 2024-02-10T08:28:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:28:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:52.766244+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:52.766502+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]': finished 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: cluster 2024-02-10T08:28:52.766682+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:52.772931+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.67:0/2797292237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:52.802618+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: cluster 2024-02-10T08:28:52.903620+0000 mgr.y (mgr.14547) 253 : cluster [DBG] pgmap v210: 520 pgs: 96 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:52.906552+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.321802+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.444321+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.768186+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.768462+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: cluster 2024-02-10T08:28:53.768525+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.777458+0000 mon.a (mon.0) 953 : audit [INF] from='client.? 172.21.15.67:0/1204513287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.779671+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[17385]: audit 2024-02-10T08:28:53.780249+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:52.766244+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:52.766502+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]': finished 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: cluster 2024-02-10T08:28:52.766682+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:52.772931+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.67:0/2797292237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:52.802618+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: cluster 2024-02-10T08:28:52.903620+0000 mgr.y (mgr.14547) 253 : cluster [DBG] pgmap v210: 520 pgs: 96 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:52.906552+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.321802+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.444321+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.768186+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.768462+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: cluster 2024-02-10T08:28:53.768525+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.777458+0000 mon.a (mon.0) 953 : audit [INF] from='client.? 172.21.15.67:0/1204513287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.779671+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:54.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:53 smithi067 bash[24488]: audit 2024-02-10T08:28:53.780249+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:52.766244+0000 mon.a (mon.0) 945 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi067-54237-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:52.766502+0000 mon.a (mon.0) 946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-7"}]': finished 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: cluster 2024-02-10T08:28:52.766682+0000 mon.a (mon.0) 947 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:52.772931+0000 mon.b (mon.1) 56 : audit [INF] from='client.? 172.21.15.67:0/2797292237' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:52.802618+0000 mon.a (mon.0) 948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: cluster 2024-02-10T08:28:52.903620+0000 mgr.y (mgr.14547) 253 : cluster [DBG] pgmap v210: 520 pgs: 96 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:52.906552+0000 mon.c (mon.2) 176 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.321802+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.444321+0000 mon.a (mon.0) 949 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.768186+0000 mon.a (mon.0) 950 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.768462+0000 mon.a (mon.0) 951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: cluster 2024-02-10T08:28:53.768525+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.777458+0000 mon.a (mon.0) 953 : audit [INF] from='client.? 172.21.15.67:0/1204513287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.779671+0000 mon.c (mon.2) 178 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:53 smithi134 bash[18722]: audit 2024-02-10T08:28:53.780249+0000 mon.a (mon.0) 954 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]: dispatch 2024-02-10T08:28:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:55 smithi067 bash[17385]: audit 2024-02-10T08:28:54.322866+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:55 smithi067 bash[24488]: audit 2024-02-10T08:28:54.322866+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:55 smithi134 bash[18722]: audit 2024-02-10T08:28:54.322866+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:56.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: cluster 2024-02-10T08:28:54.904301+0000 mgr.y (mgr.14547) 254 : cluster [DBG] pgmap v212: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-10T08:28:56.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.009089+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.67:0/1204513287' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:56.603 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.009197+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]': finished 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: cluster 2024-02-10T08:28:55.009402+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.057112+0000 mon.a (mon.0) 958 : audit [INF] from='client.? 172.21.15.67:0/4203686750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.060638+0000 mon.a (mon.0) 959 : audit [INF] from='client.? 172.21.15.67:0/1440800336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi067-54237-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.324220+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.691788+0000 mon.a (mon.0) 960 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.714733+0000 mon.a (mon.0) 961 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:55.719798+0000 mon.a (mon.0) 962 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:56.003883+0000 mon.a (mon.0) 963 : audit [INF] from='client.? 172.21.15.67:0/4203686750' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:56.003975+0000 mon.a (mon.0) 964 : audit [INF] from='client.? 172.21.15.67:0/1440800336' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi067-54237-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:56.004096+0000 mon.a (mon.0) 965 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:56.604 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: cluster 2024-02-10T08:28:56.004164+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: cluster 2024-02-10T08:28:54.904301+0000 mgr.y (mgr.14547) 254 : cluster [DBG] pgmap v212: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.009089+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.67:0/1204513287' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.009197+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]': finished 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: cluster 2024-02-10T08:28:55.009402+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.057112+0000 mon.a (mon.0) 958 : audit [INF] from='client.? 172.21.15.67:0/4203686750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.060638+0000 mon.a (mon.0) 959 : audit [INF] from='client.? 172.21.15.67:0/1440800336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi067-54237-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.324220+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.691788+0000 mon.a (mon.0) 960 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.714733+0000 mon.a (mon.0) 961 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:55.719798+0000 mon.a (mon.0) 962 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:56.605 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:56.003883+0000 mon.a (mon.0) 963 : audit [INF] from='client.? 172.21.15.67:0/4203686750' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:56.003975+0000 mon.a (mon.0) 964 : audit [INF] from='client.? 172.21.15.67:0/1440800336' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi067-54237-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:56.004096+0000 mon.a (mon.0) 965 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: cluster 2024-02-10T08:28:56.004164+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:56.010716+0000 mon.a (mon.0) 967 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi067-54461-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[24488]: audit 2024-02-10T08:28:56.012997+0000 mon.a (mon.0) 968 : audit [INF] from='client.? 172.21.15.67:0/928480465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:56.010716+0000 mon.a (mon.0) 967 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi067-54461-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:56.606 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:56 smithi067 bash[17385]: audit 2024-02-10T08:28:56.012997+0000 mon.a (mon.0) 968 : audit [INF] from='client.? 172.21.15.67:0/928480465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:56.609 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-10T08:28:56.610 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-10T08:28:56.610 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-10T08:28:56.610 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-10T08:28:56.610 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-10T08:28:56.610 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-10T08:28:56.610 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (13193 ms) 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6947 ms) 2024-02-10T08:28:56.611 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-10T08:28:56.612 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (4445 ms) 2024-02-10T08:28:56.612 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-10T08:28:56.612 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2547 ms) 2024-02-10T08:28:56.612 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-10T08:28:56.612 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (4367 ms) 2024-02-10T08:28:56.612 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-10T08:28:56.613 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4383 ms) 2024-02-10T08:28:56.613 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-10T08:28:56.613 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3800 ms) 2024-02-10T08:28:56.613 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (39683 ms total) 2024-02-10T08:28:56.613 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: 2024-02-10T08:28:56.613 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-10T08:28:56.614 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (39683 ms total) 2024-02-10T08:28:56.614 INFO:tasks.workunit.client.0.smithi067.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-10T08:28:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: cluster 2024-02-10T08:28:54.904301+0000 mgr.y (mgr.14547) 254 : cluster [DBG] pgmap v212: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-10T08:28:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.009089+0000 mon.a (mon.0) 955 : audit [INF] from='client.? 172.21.15.67:0/1204513287' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54641-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.009197+0000 mon.a (mon.0) 956 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-7"}]': finished 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: cluster 2024-02-10T08:28:55.009402+0000 mon.a (mon.0) 957 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.057112+0000 mon.a (mon.0) 958 : audit [INF] from='client.? 172.21.15.67:0/4203686750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.060638+0000 mon.a (mon.0) 959 : audit [INF] from='client.? 172.21.15.67:0/1440800336' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi067-54237-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.324220+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.691788+0000 mon.a (mon.0) 960 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.714733+0000 mon.a (mon.0) 961 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:57.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:55.719798+0000 mon.a (mon.0) 962 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:28:57.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:56.003883+0000 mon.a (mon.0) 963 : audit [INF] from='client.? 172.21.15.67:0/4203686750' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:57.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:56.003975+0000 mon.a (mon.0) 964 : audit [INF] from='client.? 172.21.15.67:0/1440800336' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi067-54237-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:57.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:56.004096+0000 mon.a (mon.0) 965 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:28:57.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: cluster 2024-02-10T08:28:56.004164+0000 mon.a (mon.0) 966 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-10T08:28:57.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:56.010716+0000 mon.a (mon.0) 967 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi067-54461-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:28:57.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:56 smithi134 bash[18722]: audit 2024-02-10T08:28:56.012997+0000 mon.a (mon.0) 968 : audit [INF] from='client.? 172.21.15.67:0/928480465' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: audit 2024-02-10T08:28:56.326965+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: audit 2024-02-10T08:28:56.601338+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4"}]: dispatch 2024-02-10T08:28:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: audit 2024-02-10T08:28:57.328183+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: audit 2024-02-10T08:28:57.772483+0000 mon.a (mon.0) 970 : audit [INF] from='client.? 172.21.15.67:0/928480465' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: audit 2024-02-10T08:28:57.772754+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4"}]': finished 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: cluster 2024-02-10T08:28:57.773058+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17385]: audit 2024-02-10T08:28:57.775698+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:28:57] "GET /metrics HTTP/1.1" 200 204699 "" "Prometheus/2.33.4" 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: audit 2024-02-10T08:28:56.326965+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: audit 2024-02-10T08:28:56.601338+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4"}]: dispatch 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: audit 2024-02-10T08:28:57.328183+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: audit 2024-02-10T08:28:57.772483+0000 mon.a (mon.0) 970 : audit [INF] from='client.? 172.21.15.67:0/928480465' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: audit 2024-02-10T08:28:57.772754+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4"}]': finished 2024-02-10T08:28:58.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: cluster 2024-02-10T08:28:57.773058+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-10T08:28:58.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:57 smithi067 bash[24488]: audit 2024-02-10T08:28:57.775698+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: audit 2024-02-10T08:28:56.326965+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: audit 2024-02-10T08:28:56.601338+0000 mon.a (mon.0) 969 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4"}]: dispatch 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: audit 2024-02-10T08:28:57.328183+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: audit 2024-02-10T08:28:57.772483+0000 mon.a (mon.0) 970 : audit [INF] from='client.? 172.21.15.67:0/928480465' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi067-54245-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: audit 2024-02-10T08:28:57.772754+0000 mon.a (mon.0) 971 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4"}]': finished 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: cluster 2024-02-10T08:28:57.773058+0000 mon.a (mon.0) 972 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-10T08:28:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:57 smithi134 bash[18722]: audit 2024-02-10T08:28:57.775698+0000 mon.c (mon.2) 183 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: cluster 2024-02-10T08:28:56.906239+0000 mgr.y (mgr.14547) 255 : cluster [DBG] pgmap v215: 456 pgs: 32 creating+peering, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:57.781152+0000 mon.a (mon.0) 973 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]: dispatch 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:57.814187+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.329300+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: cluster 2024-02-10T08:28:58.437624+0000 mon.a (mon.0) 975 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.455061+0000 mon.a (mon.0) 976 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi067-54461-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.455154+0000 mon.a (mon.0) 977 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]': finished 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.455335+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: cluster 2024-02-10T08:28:58.455492+0000 mon.a (mon.0) 979 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.467964+0000 mon.a (mon.0) 980 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.476497+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.67:0/1767911909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.484690+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.506406+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[17385]: audit 2024-02-10T08:28:58.507704+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: cluster 2024-02-10T08:28:56.906239+0000 mgr.y (mgr.14547) 255 : cluster [DBG] pgmap v215: 456 pgs: 32 creating+peering, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:57.781152+0000 mon.a (mon.0) 973 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:57.814187+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.329300+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: cluster 2024-02-10T08:28:58.437624+0000 mon.a (mon.0) 975 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.455061+0000 mon.a (mon.0) 976 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi067-54461-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.455154+0000 mon.a (mon.0) 977 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]': finished 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.455335+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: cluster 2024-02-10T08:28:58.455492+0000 mon.a (mon.0) 979 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.467964+0000 mon.a (mon.0) 980 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.476497+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.67:0/1767911909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.484690+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.506406+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:59.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:28:59 smithi067 bash[24488]: audit 2024-02-10T08:28:58.507704+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: cluster 2024-02-10T08:28:56.906239+0000 mgr.y (mgr.14547) 255 : cluster [DBG] pgmap v215: 456 pgs: 32 creating+peering, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-02-10T08:28:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:57.781152+0000 mon.a (mon.0) 973 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:57.814187+0000 mon.a (mon.0) 974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.329300+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: cluster 2024-02-10T08:28:58.437624+0000 mon.a (mon.0) 975 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.455061+0000 mon.a (mon.0) 976 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi067-54461-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.455154+0000 mon.a (mon.0) 977 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6"}]': finished 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.455335+0000 mon.a (mon.0) 978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: cluster 2024-02-10T08:28:58.455492+0000 mon.a (mon.0) 979 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.467964+0000 mon.a (mon.0) 980 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.476497+0000 mon.c (mon.2) 185 : audit [INF] from='client.? 172.21.15.67:0/1767911909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.484690+0000 mon.a (mon.0) 981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.506406+0000 mon.c (mon.2) 186 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:28:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:28:59 smithi134 bash[18722]: audit 2024-02-10T08:28:58.507704+0000 mon.a (mon.0) 982 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:00.180 INFO:tasks.workunit.client.0.smithi067.stdout:_watch_notify_pp: flushed 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3021 ms) 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: trying... 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422137360 notify_id 365072220163 notifier_gid 15222 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: timed out 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: flushing 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: flushed 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3030 ms) 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: List watches 2024-02-10T08:29:00.181 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify2 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422137360 notify_id 377957122051 notifier_gid 15222 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify2 done 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: watch_check 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: unwatch2 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: flushing 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: done 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3014 ms) 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: List watches 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify2 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: handle_notify cookie 94003422137360 notify_id 390842023940 notifier_gid 15222 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: notify2 done 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: watch_check 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: unwatch2 2024-02-10T08:29:00.182 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: flushing 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: done 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3013 ms) 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16313 ms total) 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (43295 ms total) 2024-02-10T08:29:00.183 INFO:tasks.workunit.client.0.smithi067.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: cluster 2024-02-10T08:28:58.907205+0000 mgr.y (mgr.14547) 256 : cluster [DBG] pgmap v218: 464 pgs: 72 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.330363+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.454413+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.454598+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.454781+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: cluster 2024-02-10T08:28:59.458554+0000 mon.a (mon.0) 986 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.467440+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.468237+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.474686+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.67:0/2773383969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:00 smithi134 bash[18722]: audit 2024-02-10T08:28:59.478886+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: cluster 2024-02-10T08:28:58.907205+0000 mgr.y (mgr.14547) 256 : cluster [DBG] pgmap v218: 464 pgs: 72 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.330363+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.454413+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.454598+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.454781+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: cluster 2024-02-10T08:28:59.458554+0000 mon.a (mon.0) 986 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.467440+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.468237+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.474686+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.67:0/2773383969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[17385]: audit 2024-02-10T08:28:59.478886+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: cluster 2024-02-10T08:28:58.907205+0000 mgr.y (mgr.14547) 256 : cluster [DBG] pgmap v218: 464 pgs: 72 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.330363+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.454413+0000 mon.a (mon.0) 983 : audit [INF] from='client.? 172.21.15.67:0/2740353724' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi067-54538-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.454598+0000 mon.a (mon.0) 984 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi067-54237-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.454781+0000 mon.a (mon.0) 985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: cluster 2024-02-10T08:28:59.458554+0000 mon.a (mon.0) 986 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.467440+0000 mon.c (mon.2) 188 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.468237+0000 mon.a (mon.0) 987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.474686+0000 mon.c (mon.2) 189 : audit [INF] from='client.? 172.21.15.67:0/2773383969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:00 smithi067 bash[24488]: audit 2024-02-10T08:28:59.478886+0000 mon.a (mon.0) 988 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.146682+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]': finished 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.146852+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: cluster 2024-02-10T08:29:00.147124+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.167083+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]: dispatch 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.169694+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.67:0/1804120055' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.185513+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]: dispatch 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.199922+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.333216+0000 mon.c (mon.2) 192 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:00.788544+0000 mon.a (mon.0) 994 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: cluster 2024-02-10T08:29:01.143771+0000 mon.a (mon.0) 995 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:01.152607+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]': finished 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:01.152708+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: cluster 2024-02-10T08:29:01.152865+0000 mon.a (mon.0) 998 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:01.168160+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.67:0/3620292065' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[17385]: audit 2024-02-10T08:29:01.168915+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.146682+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]': finished 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.146852+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: cluster 2024-02-10T08:29:00.147124+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.167083+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]: dispatch 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.169694+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.67:0/1804120055' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.185513+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]: dispatch 2024-02-10T08:29:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.199922+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.333216+0000 mon.c (mon.2) 192 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:00.788544+0000 mon.a (mon.0) 994 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: cluster 2024-02-10T08:29:01.143771+0000 mon.a (mon.0) 995 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:01.152607+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]': finished 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:01.152708+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: cluster 2024-02-10T08:29:01.152865+0000 mon.a (mon.0) 998 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:01.168160+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.67:0/3620292065' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:01 smithi067 bash[24488]: audit 2024-02-10T08:29:01.168915+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:29:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.146682+0000 mon.a (mon.0) 989 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-9"}]': finished 2024-02-10T08:29:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.146852+0000 mon.a (mon.0) 990 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi067-54245-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: cluster 2024-02-10T08:29:00.147124+0000 mon.a (mon.0) 991 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-10T08:29:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.167083+0000 mon.c (mon.2) 190 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]: dispatch 2024-02-10T08:29:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.169694+0000 mon.c (mon.2) 191 : audit [INF] from='client.? 172.21.15.67:0/1804120055' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.185513+0000 mon.a (mon.0) 992 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]: dispatch 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.199922+0000 mon.a (mon.0) 993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.333216+0000 mon.c (mon.2) 192 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:00.788544+0000 mon.a (mon.0) 994 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: cluster 2024-02-10T08:29:01.143771+0000 mon.a (mon.0) 995 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:01.152607+0000 mon.a (mon.0) 996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-9", "mode": "writeback"}]': finished 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:01.152708+0000 mon.a (mon.0) 997 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: cluster 2024-02-10T08:29:01.152865+0000 mon.a (mon.0) 998 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:01.168160+0000 mon.c (mon.2) 193 : audit [INF] from='client.? 172.21.15.67:0/3620292065' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:01 smithi134 bash[18722]: audit 2024-02-10T08:29:01.168915+0000 mon.a (mon.0) 999 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: cluster 2024-02-10T08:29:00.907824+0000 mgr.y (mgr.14547) 257 : cluster [DBG] pgmap v221: 428 pgs: 104 unknown, 324 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:01.340228+0000 mon.c (mon.2) 194 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:01.382623+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:01.875175+0000 mgr.y (mgr.14547) 258 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:02.344443+0000 mon.c (mon.2) 196 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:02.367426+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: cluster 2024-02-10T08:29:02.367820+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:02.371724+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.67:0/1962487886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:02.410665+0000 mon.a (mon.0) 1002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[17385]: audit 2024-02-10T08:29:02.412639+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: cluster 2024-02-10T08:29:00.907824+0000 mgr.y (mgr.14547) 257 : cluster [DBG] pgmap v221: 428 pgs: 104 unknown, 324 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:01.340228+0000 mon.c (mon.2) 194 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:01.382623+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:01.875175+0000 mgr.y (mgr.14547) 258 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:02.344443+0000 mon.c (mon.2) 196 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:02.367426+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: cluster 2024-02-10T08:29:02.367820+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:02.371724+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.67:0/1962487886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:02.410665+0000 mon.a (mon.0) 1002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:02 smithi067 bash[24488]: audit 2024-02-10T08:29:02.412639+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:03.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: cluster 2024-02-10T08:29:00.907824+0000 mgr.y (mgr.14547) 257 : cluster [DBG] pgmap v221: 428 pgs: 104 unknown, 324 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:01.340228+0000 mon.c (mon.2) 194 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:01.382623+0000 mon.c (mon.2) 195 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:01.875175+0000 mgr.y (mgr.14547) 258 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:02.344443+0000 mon.c (mon.2) 196 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:02.367426+0000 mon.a (mon.0) 1000 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi067-54237-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: cluster 2024-02-10T08:29:02.367820+0000 mon.a (mon.0) 1001 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:02.371724+0000 mon.b (mon.1) 57 : audit [INF] from='client.? 172.21.15.67:0/1962487886' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:02.410665+0000 mon.a (mon.0) 1002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:02 smithi134 bash[18722]: audit 2024-02-10T08:29:02.412639+0000 mon.a (mon.0) 1003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:29:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.035168+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.041348+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.042619+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.045436+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.046806+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.049352+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:29:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.051059+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.055638+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.057521+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.058363+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.346208+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.374647+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.374883+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: cluster 2024-02-10T08:29:03.375039+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.377496+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:04.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:04 smithi134 bash[18722]: audit 2024-02-10T08:29:03.378972+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.035168+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.041348+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.042619+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.045436+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.046806+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.049352+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.051059+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.055638+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.057521+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.058363+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.346208+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.374647+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.374883+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: cluster 2024-02-10T08:29:03.375039+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.377496+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[17385]: audit 2024-02-10T08:29:03.378972+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.035168+0000 mon.c (mon.2) 197 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.041348+0000 mon.c (mon.2) 198 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.042619+0000 mon.c (mon.2) 199 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.045436+0000 mon.c (mon.2) 200 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.046806+0000 mon.c (mon.2) 201 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.049352+0000 mon.c (mon.2) 202 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.051059+0000 mon.c (mon.2) 203 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:29:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.055638+0000 mon.c (mon.2) 204 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.057521+0000 mon.c (mon.2) 205 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.058363+0000 mon.c (mon.2) 206 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.346208+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.374647+0000 mon.a (mon.0) 1004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi067-54245-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.374883+0000 mon.a (mon.0) 1005 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: cluster 2024-02-10T08:29:03.375039+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.377496+0000 mon.c (mon.2) 208 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:04.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:04 smithi067 bash[24488]: audit 2024-02-10T08:29:03.378972+0000 mon.a (mon.0) 1007 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:02.910783+0000 mgr.y (mgr.14547) 259 : cluster [DBG] pgmap v224: 492 pgs: 64 unknown, 32 creating+peering, 396 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.036566+0000 mgr.y (mgr.14547) 260 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.041536+0000 mgr.y (mgr.14547) 261 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.042936+0000 mgr.y (mgr.14547) 262 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.045806+0000 mgr.y (mgr.14547) 263 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.047985+0000 mgr.y (mgr.14547) 264 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.050273+0000 mgr.y (mgr.14547) 265 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.051340+0000 mgr.y (mgr.14547) 266 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.055998+0000 mgr.y (mgr.14547) 267 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.057670+0000 mgr.y (mgr.14547) 268 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:03.058601+0000 mgr.y (mgr.14547) 269 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:29:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:03.264318+0000 osd.0 (osd.0) 9 : cluster [DBG] 16.0 deep-scrub starts 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:03.323683+0000 osd.0 (osd.0) 10 : cluster [DBG] 16.0 deep-scrub ok 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:03.724656+0000 osd.6 (osd.6) 5 : cluster [DBG] 16.6 deep-scrub starts 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:03.763994+0000 osd.6 (osd.6) 6 : cluster [DBG] 16.6 deep-scrub ok 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.000081+0000 mon.a (mon.0) 1008 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.116898+0000 osd.3 (osd.3) 7 : cluster [DBG] 16.4 deep-scrub starts 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.120808+0000 osd.3 (osd.3) 8 : cluster [DBG] 16.4 deep-scrub ok 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:04.351245+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.366459+0000 mon.a (mon.0) 1009 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:04.373043+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]': finished 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.373222+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:04.393123+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.67:0/418612728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:05.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[17385]: audit 2024-02-10T08:29:04.393812+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:02.910783+0000 mgr.y (mgr.14547) 259 : cluster [DBG] pgmap v224: 492 pgs: 64 unknown, 32 creating+peering, 396 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.036566+0000 mgr.y (mgr.14547) 260 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.041536+0000 mgr.y (mgr.14547) 261 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.042936+0000 mgr.y (mgr.14547) 262 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.045806+0000 mgr.y (mgr.14547) 263 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.047985+0000 mgr.y (mgr.14547) 264 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.050273+0000 mgr.y (mgr.14547) 265 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.051340+0000 mgr.y (mgr.14547) 266 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.055998+0000 mgr.y (mgr.14547) 267 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.057670+0000 mgr.y (mgr.14547) 268 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:03.058601+0000 mgr.y (mgr.14547) 269 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:29:05.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:03.264318+0000 osd.0 (osd.0) 9 : cluster [DBG] 16.0 deep-scrub starts 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:03.323683+0000 osd.0 (osd.0) 10 : cluster [DBG] 16.0 deep-scrub ok 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:03.724656+0000 osd.6 (osd.6) 5 : cluster [DBG] 16.6 deep-scrub starts 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:03.763994+0000 osd.6 (osd.6) 6 : cluster [DBG] 16.6 deep-scrub ok 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.000081+0000 mon.a (mon.0) 1008 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.116898+0000 osd.3 (osd.3) 7 : cluster [DBG] 16.4 deep-scrub starts 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.120808+0000 osd.3 (osd.3) 8 : cluster [DBG] 16.4 deep-scrub ok 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:04.351245+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.366459+0000 mon.a (mon.0) 1009 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:04.373043+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]': finished 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.373222+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:04.393123+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.67:0/418612728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:05.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:05 smithi067 bash[24488]: audit 2024-02-10T08:29:04.393812+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:02.910783+0000 mgr.y (mgr.14547) 259 : cluster [DBG] pgmap v224: 492 pgs: 64 unknown, 32 creating+peering, 396 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.036566+0000 mgr.y (mgr.14547) 260 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.0"}]: dispatch 2024-02-10T08:29:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.041536+0000 mgr.y (mgr.14547) 261 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.1"}]: dispatch 2024-02-10T08:29:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.042936+0000 mgr.y (mgr.14547) 262 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.2"}]: dispatch 2024-02-10T08:29:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.045806+0000 mgr.y (mgr.14547) 263 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.3"}]: dispatch 2024-02-10T08:29:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.047985+0000 mgr.y (mgr.14547) 264 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.4"}]: dispatch 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.050273+0000 mgr.y (mgr.14547) 265 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.5"}]: dispatch 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.051340+0000 mgr.y (mgr.14547) 266 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.6"}]: dispatch 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.055998+0000 mgr.y (mgr.14547) 267 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.7"}]: dispatch 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.057670+0000 mgr.y (mgr.14547) 268 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.8"}]: dispatch 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:03.058601+0000 mgr.y (mgr.14547) 269 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "16.9"}]: dispatch 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:03.264318+0000 osd.0 (osd.0) 9 : cluster [DBG] 16.0 deep-scrub starts 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:03.323683+0000 osd.0 (osd.0) 10 : cluster [DBG] 16.0 deep-scrub ok 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:03.724656+0000 osd.6 (osd.6) 5 : cluster [DBG] 16.6 deep-scrub starts 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:03.763994+0000 osd.6 (osd.6) 6 : cluster [DBG] 16.6 deep-scrub ok 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.000081+0000 mon.a (mon.0) 1008 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.116898+0000 osd.3 (osd.3) 7 : cluster [DBG] 16.4 deep-scrub starts 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.120808+0000 osd.3 (osd.3) 8 : cluster [DBG] 16.4 deep-scrub ok 2024-02-10T08:29:05.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:04.351245+0000 mon.c (mon.2) 209 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:05.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.366459+0000 mon.a (mon.0) 1009 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:29:05.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:04.373043+0000 mon.a (mon.0) 1010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-9"}]': finished 2024-02-10T08:29:05.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.373222+0000 mon.a (mon.0) 1011 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-10T08:29:05.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:04.393123+0000 mon.c (mon.2) 210 : audit [INF] from='client.? 172.21.15.67:0/418612728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:05.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:05 smithi134 bash[18722]: audit 2024-02-10T08:29:04.393812+0000 mon.a (mon.0) 1012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:06.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.544946+0000 osd.4 (osd.4) 5 : cluster [DBG] 16.2 deep-scrub starts 2024-02-10T08:29:06.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.546742+0000 osd.4 (osd.4) 6 : cluster [DBG] 16.2 deep-scrub ok 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[24488]: audit 2024-02-10T08:29:05.357048+0000 mon.c (mon.2) 211 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[24488]: audit 2024-02-10T08:29:05.684548+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[24488]: cluster 2024-02-10T08:29:05.684750+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[24488]: audit 2024-02-10T08:29:05.709927+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.67:0/1941164268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.544946+0000 osd.4 (osd.4) 5 : cluster [DBG] 16.2 deep-scrub starts 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.546742+0000 osd.4 (osd.4) 6 : cluster [DBG] 16.2 deep-scrub ok 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[17385]: audit 2024-02-10T08:29:05.357048+0000 mon.c (mon.2) 211 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[17385]: audit 2024-02-10T08:29:05.684548+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[17385]: cluster 2024-02-10T08:29:05.684750+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-10T08:29:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:06 smithi067 bash[17385]: audit 2024-02-10T08:29:05.709927+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.67:0/1941164268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:06.719 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-10T08:29:06.719 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] Global test environment set-up. 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: 2024-02-10T08:28:17.824+0000 7f3b7bd49540 -1 failed for service _ceph-mon._tcp 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: 2024-02-10T08:28:17.824+0000 7f3b7bd49540 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (1014 ms) 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2012 ms) 2024-02-10T08:29:06.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (3026 ms total) 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: started 0x56227b0b8010 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: started 0x56227b0c14f0 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: started 2 aios 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: waiting 0x56227b0b8010 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: waiting 0x56227b0c14f0 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: done. 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (14380 ms) 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (14380 ms total) 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-10T08:29:06.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.Exec (5 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (6 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5476 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (22540 ms) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] 8 tests from LibRadosMisc (28030 ms total) 2024-02-10T08:29:06.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: 2024-02-10T08:29:06.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [----------] Global test environment tear-down 2024-02-10T08:29:06.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (49890 ms total) 2024-02-10T08:29:06.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-10T08:29:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:06 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.544946+0000 osd.4 (osd.4) 5 : cluster [DBG] 16.2 deep-scrub starts 2024-02-10T08:29:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:06 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.546742+0000 osd.4 (osd.4) 6 : cluster [DBG] 16.2 deep-scrub ok 2024-02-10T08:29:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:06 smithi134 bash[18722]: audit 2024-02-10T08:29:05.357048+0000 mon.c (mon.2) 211 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:06 smithi134 bash[18722]: audit 2024-02-10T08:29:05.684548+0000 mon.a (mon.0) 1013 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi067-54237-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:06 smithi134 bash[18722]: cluster 2024-02-10T08:29:05.684750+0000 mon.a (mon.0) 1014 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-10T08:29:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:06 smithi134 bash[18722]: audit 2024-02-10T08:29:05.709927+0000 mon.b (mon.1) 58 : audit [INF] from='client.? 172.21.15.67:0/1941164268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: cluster 2024-02-10T08:29:04.911409+0000 mgr.y (mgr.14547) 270 : cluster [DBG] pgmap v227: 396 pgs: 32 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: cluster 2024-02-10T08:29:05.267131+0000 osd.0 (osd.0) 11 : cluster [DBG] 16.5 deep-scrub starts 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: cluster 2024-02-10T08:29:05.269470+0000 osd.0 (osd.0) 12 : cluster [DBG] 16.5 deep-scrub ok 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: cluster 2024-02-10T08:29:05.873112+0000 osd.1 (osd.1) 5 : cluster [DBG] 16.9 deep-scrub starts 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: cluster 2024-02-10T08:29:05.940422+0000 osd.1 (osd.1) 6 : cluster [DBG] 16.9 deep-scrub ok 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:06.265826+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:06.278342+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:06.358470+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:06.690495+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:06.690903+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: cluster 2024-02-10T08:29:06.691083+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:06.702900+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:07 smithi134 bash[18722]: audit 2024-02-10T08:29:07.168592+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:07] "GET /metrics HTTP/1.1" 200 200500 "" "Prometheus/2.33.4" 2024-02-10T08:29:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: cluster 2024-02-10T08:29:04.911409+0000 mgr.y (mgr.14547) 270 : cluster [DBG] pgmap v227: 396 pgs: 32 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-10T08:29:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: cluster 2024-02-10T08:29:05.267131+0000 osd.0 (osd.0) 11 : cluster [DBG] 16.5 deep-scrub starts 2024-02-10T08:29:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: cluster 2024-02-10T08:29:05.269470+0000 osd.0 (osd.0) 12 : cluster [DBG] 16.5 deep-scrub ok 2024-02-10T08:29:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: cluster 2024-02-10T08:29:05.873112+0000 osd.1 (osd.1) 5 : cluster [DBG] 16.9 deep-scrub starts 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: cluster 2024-02-10T08:29:05.940422+0000 osd.1 (osd.1) 6 : cluster [DBG] 16.9 deep-scrub ok 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:06.265826+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:06.278342+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:06.358470+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:06.690495+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:06.690903+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: cluster 2024-02-10T08:29:06.691083+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:06.702900+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[24488]: audit 2024-02-10T08:29:07.168592+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: cluster 2024-02-10T08:29:04.911409+0000 mgr.y (mgr.14547) 270 : cluster [DBG] pgmap v227: 396 pgs: 32 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-10T08:29:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: cluster 2024-02-10T08:29:05.267131+0000 osd.0 (osd.0) 11 : cluster [DBG] 16.5 deep-scrub starts 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: cluster 2024-02-10T08:29:05.269470+0000 osd.0 (osd.0) 12 : cluster [DBG] 16.5 deep-scrub ok 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: cluster 2024-02-10T08:29:05.873112+0000 osd.1 (osd.1) 5 : cluster [DBG] 16.9 deep-scrub starts 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: cluster 2024-02-10T08:29:05.940422+0000 osd.1 (osd.1) 6 : cluster [DBG] 16.9 deep-scrub ok 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:06.265826+0000 mon.a (mon.0) 1015 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:06.278342+0000 mon.a (mon.0) 1016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:06.358470+0000 mon.c (mon.2) 212 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:06.690495+0000 mon.a (mon.0) 1017 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54357-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:06.690903+0000 mon.a (mon.0) 1018 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi067-54245-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: cluster 2024-02-10T08:29:06.691083+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:06.702900+0000 mon.c (mon.2) 213 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:08.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:07 smithi067 bash[17385]: audit 2024-02-10T08:29:07.168592+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: cluster 2024-02-10T08:29:06.912319+0000 mgr.y (mgr.14547) 271 : cluster [DBG] pgmap v230: 396 pgs: 64 creating+peering, 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 1 op/s 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: audit 2024-02-10T08:29:07.359604+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: audit 2024-02-10T08:29:07.700448+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: cluster 2024-02-10T08:29:07.700815+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: audit 2024-02-10T08:29:07.707329+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.67:0/2138264687' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: audit 2024-02-10T08:29:07.708582+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: audit 2024-02-10T08:29:07.716042+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T08:29:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:08 smithi134 bash[18722]: audit 2024-02-10T08:29:08.361181+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:09.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: cluster 2024-02-10T08:29:06.912319+0000 mgr.y (mgr.14547) 271 : cluster [DBG] pgmap v230: 396 pgs: 64 creating+peering, 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 1 op/s 2024-02-10T08:29:09.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: audit 2024-02-10T08:29:07.359604+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: audit 2024-02-10T08:29:07.700448+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: cluster 2024-02-10T08:29:07.700815+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: audit 2024-02-10T08:29:07.707329+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.67:0/2138264687' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: audit 2024-02-10T08:29:07.708582+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: audit 2024-02-10T08:29:07.716042+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[17385]: audit 2024-02-10T08:29:08.361181+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: cluster 2024-02-10T08:29:06.912319+0000 mgr.y (mgr.14547) 271 : cluster [DBG] pgmap v230: 396 pgs: 64 creating+peering, 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 1 op/s 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: audit 2024-02-10T08:29:07.359604+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: audit 2024-02-10T08:29:07.700448+0000 mon.a (mon.0) 1021 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: cluster 2024-02-10T08:29:07.700815+0000 mon.a (mon.0) 1022 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: audit 2024-02-10T08:29:07.707329+0000 mon.c (mon.2) 215 : audit [INF] from='client.? 172.21.15.67:0/2138264687' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: audit 2024-02-10T08:29:07.708582+0000 mon.a (mon.0) 1023 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: audit 2024-02-10T08:29:07.716042+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache", "force_nonempty":""}]: dispatch 2024-02-10T08:29:09.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:08 smithi067 bash[24488]: audit 2024-02-10T08:29:08.361181+0000 mon.c (mon.2) 216 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: cluster 2024-02-10T08:29:08.072904+0000 osd.2 (osd.2) 5 : cluster [DBG] 16.1 deep-scrub starts 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: cluster 2024-02-10T08:29:08.078165+0000 osd.2 (osd.2) 6 : cluster [DBG] 16.1 deep-scrub ok 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.707140+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.707490+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache", "force_nonempty":""}]': finished 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: cluster 2024-02-10T08:29:08.707832+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.718520+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54357-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.720365+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.730039+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.67:0/3403792723' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.731374+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.758657+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:08.760063+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: cluster 2024-02-10T08:29:09.056931+0000 osd.3 (osd.3) 9 : cluster [DBG] 16.7 deep-scrub starts 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: cluster 2024-02-10T08:29:09.060591+0000 osd.3 (osd.3) 10 : cluster [DBG] 16.7 deep-scrub ok 2024-02-10T08:29:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:09 smithi134 bash[18722]: audit 2024-02-10T08:29:09.363042+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: cluster 2024-02-10T08:29:08.072904+0000 osd.2 (osd.2) 5 : cluster [DBG] 16.1 deep-scrub starts 2024-02-10T08:29:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: cluster 2024-02-10T08:29:08.078165+0000 osd.2 (osd.2) 6 : cluster [DBG] 16.1 deep-scrub ok 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.707140+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.707490+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache", "force_nonempty":""}]': finished 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: cluster 2024-02-10T08:29:08.707832+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.718520+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54357-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.720365+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.730039+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.67:0/3403792723' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.731374+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.758657+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:08.760063+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: cluster 2024-02-10T08:29:09.056931+0000 osd.3 (osd.3) 9 : cluster [DBG] 16.7 deep-scrub starts 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: cluster 2024-02-10T08:29:09.060591+0000 osd.3 (osd.3) 10 : cluster [DBG] 16.7 deep-scrub ok 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[17385]: audit 2024-02-10T08:29:09.363042+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: cluster 2024-02-10T08:29:08.072904+0000 osd.2 (osd.2) 5 : cluster [DBG] 16.1 deep-scrub starts 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: cluster 2024-02-10T08:29:08.078165+0000 osd.2 (osd.2) 6 : cluster [DBG] 16.1 deep-scrub ok 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.707140+0000 mon.a (mon.0) 1025 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi067-54237-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.707490+0000 mon.a (mon.0) 1026 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache", "force_nonempty":""}]': finished 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: cluster 2024-02-10T08:29:08.707832+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.718520+0000 mon.a (mon.0) 1028 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54357-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.720365+0000 mon.a (mon.0) 1029 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.730039+0000 mon.c (mon.2) 217 : audit [INF] from='client.? 172.21.15.67:0/3403792723' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.731374+0000 mon.a (mon.0) 1030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.758657+0000 mon.c (mon.2) 218 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:08.760063+0000 mon.a (mon.0) 1031 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: cluster 2024-02-10T08:29:09.056931+0000 osd.3 (osd.3) 9 : cluster [DBG] 16.7 deep-scrub starts 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: cluster 2024-02-10T08:29:09.060591+0000 osd.3 (osd.3) 10 : cluster [DBG] 16.7 deep-scrub ok 2024-02-10T08:29:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:09 smithi067 bash[24488]: audit 2024-02-10T08:29:09.363042+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:11.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: cluster 2024-02-10T08:29:08.913361+0000 mgr.y (mgr.14547) 272 : cluster [DBG] pgmap v233: 452 pgs: 32 creating+peering, 128 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T08:29:11.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: cluster 2024-02-10T08:29:09.293422+0000 osd.0 (osd.0) 13 : cluster [DBG] 16.8 deep-scrub starts 2024-02-10T08:29:11.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: cluster 2024-02-10T08:29:09.297162+0000 osd.0 (osd.0) 14 : cluster [DBG] 16.8 deep-scrub ok 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: cluster 2024-02-10T08:29:09.701989+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.709452+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54357-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.709836+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.710740+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.710970+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: cluster 2024-02-10T08:29:09.711096+0000 mon.a (mon.0) 1037 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.713233+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.713857+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.715317+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:09.735824+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache"}]: dispatch 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:10.364661+0000 mon.c (mon.2) 221 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:10.692653+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:29:11.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:10.693055+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]': finished 2024-02-10T08:29:11.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:10.693243+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache"}]': finished 2024-02-10T08:29:11.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: cluster 2024-02-10T08:29:10.694311+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-10T08:29:11.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:10.701844+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]: dispatch 2024-02-10T08:29:11.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:10 smithi134 bash[18722]: audit 2024-02-10T08:29:10.703764+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]: dispatch 2024-02-10T08:29:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: cluster 2024-02-10T08:29:08.913361+0000 mgr.y (mgr.14547) 272 : cluster [DBG] pgmap v233: 452 pgs: 32 creating+peering, 128 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: cluster 2024-02-10T08:29:09.293422+0000 osd.0 (osd.0) 13 : cluster [DBG] 16.8 deep-scrub starts 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: cluster 2024-02-10T08:29:09.297162+0000 osd.0 (osd.0) 14 : cluster [DBG] 16.8 deep-scrub ok 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: cluster 2024-02-10T08:29:09.701989+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.709452+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54357-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.709836+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.710740+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.710970+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: cluster 2024-02-10T08:29:09.711096+0000 mon.a (mon.0) 1037 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.713233+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.713857+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:29:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.715317+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:09.735824+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache"}]: dispatch 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:10.364661+0000 mon.c (mon.2) 221 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:10.692653+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: cluster 2024-02-10T08:29:08.913361+0000 mgr.y (mgr.14547) 272 : cluster [DBG] pgmap v233: 452 pgs: 32 creating+peering, 128 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: cluster 2024-02-10T08:29:09.293422+0000 osd.0 (osd.0) 13 : cluster [DBG] 16.8 deep-scrub starts 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: cluster 2024-02-10T08:29:09.297162+0000 osd.0 (osd.0) 14 : cluster [DBG] 16.8 deep-scrub ok 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: cluster 2024-02-10T08:29:09.701989+0000 mon.a (mon.0) 1032 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.709452+0000 mon.a (mon.0) 1033 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54357-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.709836+0000 mon.a (mon.0) 1034 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.710740+0000 mon.a (mon.0) 1035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.710970+0000 mon.a (mon.0) 1036 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: cluster 2024-02-10T08:29:09.711096+0000 mon.a (mon.0) 1037 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.713233+0000 mon.c (mon.2) 220 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.713857+0000 mon.a (mon.0) 1038 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]: dispatch 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.715317+0000 mon.a (mon.0) 1039 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:09.735824+0000 mon.a (mon.0) 1040 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache"}]: dispatch 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:10.364661+0000 mon.c (mon.2) 221 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:10.692653+0000 mon.a (mon.0) 1041 : audit [INF] from='client.? 172.21.15.67:0/2472815256' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi067-54461-9"}]': finished 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:10.693055+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]': finished 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:10.693243+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache"}]': finished 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: cluster 2024-02-10T08:29:10.694311+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:10.701844+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]: dispatch 2024-02-10T08:29:11.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[24488]: audit 2024-02-10T08:29:10.703764+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]: dispatch 2024-02-10T08:29:11.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:10.693055+0000 mon.a (mon.0) 1042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-11"}]': finished 2024-02-10T08:29:11.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:10.693243+0000 mon.a (mon.0) 1043 : audit [INF] from='client.? 172.21.15.67:0/2140186097' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54357-10", "tierpool":"test-rados-api-smithi067-54357-10-cache"}]': finished 2024-02-10T08:29:11.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: cluster 2024-02-10T08:29:10.694311+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-10T08:29:11.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:10.701844+0000 mon.c (mon.2) 222 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]: dispatch 2024-02-10T08:29:11.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:10 smithi067 bash[17385]: audit 2024-02-10T08:29:10.703764+0000 mon.a (mon.0) 1045 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]: dispatch 2024-02-10T08:29:12.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:29:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: cluster 2024-02-10T08:29:10.648915+0000 osd.5 (osd.5) 5 : cluster [DBG] 16.3 deep-scrub starts 2024-02-10T08:29:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:10.709174+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.67:0/2777094699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:10.730343+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:10.743036+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: cluster 2024-02-10T08:29:10.782702+0000 osd.5 (osd.5) 6 : cluster [DBG] 16.3 deep-scrub ok 2024-02-10T08:29:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: cluster 2024-02-10T08:29:10.914308+0000 mgr.y (mgr.14547) 273 : cluster [DBG] pgmap v236: 420 pgs: 32 creating+peering, 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.199382+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.200505+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.203235+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.204212+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.204826+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.366340+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: cluster 2024-02-10T08:29:11.684440+0000 mon.a (mon.0) 1050 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.694373+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]': finished 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.694936+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.695125+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: cluster 2024-02-10T08:29:11.695232+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.709236+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.709874+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:11 smithi134 bash[18722]: audit 2024-02-10T08:29:11.711978+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.67:0/2450805515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi067-54245-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: cluster 2024-02-10T08:29:10.648915+0000 osd.5 (osd.5) 5 : cluster [DBG] 16.3 deep-scrub starts 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:10.709174+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.67:0/2777094699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:10.730343+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:10.743036+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: cluster 2024-02-10T08:29:10.782702+0000 osd.5 (osd.5) 6 : cluster [DBG] 16.3 deep-scrub ok 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: cluster 2024-02-10T08:29:10.914308+0000 mgr.y (mgr.14547) 273 : cluster [DBG] pgmap v236: 420 pgs: 32 creating+peering, 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.199382+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.200505+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.203235+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.204212+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.204826+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.366340+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: cluster 2024-02-10T08:29:11.684440+0000 mon.a (mon.0) 1050 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.694373+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]': finished 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.694936+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.695125+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: cluster 2024-02-10T08:29:11.695232+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.709236+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.709874+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[17385]: audit 2024-02-10T08:29:11.711978+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.67:0/2450805515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi067-54245-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: cluster 2024-02-10T08:29:10.648915+0000 osd.5 (osd.5) 5 : cluster [DBG] 16.3 deep-scrub starts 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:10.709174+0000 mon.c (mon.2) 223 : audit [INF] from='client.? 172.21.15.67:0/2777094699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:10.730343+0000 mon.a (mon.0) 1046 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:10.743036+0000 mon.c (mon.2) 224 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: cluster 2024-02-10T08:29:10.782702+0000 osd.5 (osd.5) 6 : cluster [DBG] 16.3 deep-scrub ok 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: cluster 2024-02-10T08:29:10.914308+0000 mgr.y (mgr.14547) 273 : cluster [DBG] pgmap v236: 420 pgs: 32 creating+peering, 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.199382+0000 mon.a (mon.0) 1047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.200505+0000 mon.c (mon.2) 225 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.203235+0000 mon.a (mon.0) 1048 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.204212+0000 mon.c (mon.2) 226 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.204826+0000 mon.a (mon.0) 1049 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.366340+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: cluster 2024-02-10T08:29:11.684440+0000 mon.a (mon.0) 1050 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.694373+0000 mon.a (mon.0) 1051 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-11", "mode": "writeback"}]': finished 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.694936+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi067-54237-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.695125+0000 mon.a (mon.0) 1053 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:12.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: cluster 2024-02-10T08:29:11.695232+0000 mon.a (mon.0) 1054 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-10T08:29:12.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.709236+0000 mon.c (mon.2) 228 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.709874+0000 mon.a (mon.0) 1055 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:12.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:11 smithi067 bash[24488]: audit 2024-02-10T08:29:11.711978+0000 mon.a (mon.0) 1056 : audit [INF] from='client.? 172.21.15.67:0/2450805515' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi067-54245-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:13.170 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: Running main() from gmock_main.cc 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [----------] Global test environment set-up. 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolList (14835 ms) 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6397 ms) 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4406 ms) 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2762 ms) 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3211 ms) 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-10T08:29:13.171 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5344 ms) 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5683 ms) 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4905 ms) 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8788 ms) 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [----------] 9 tests from LibRadosPools (56331 ms total) 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [----------] Global test environment tear-down 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (56331 ms total) 2024-02-10T08:29:13.172 INFO:tasks.workunit.client.0.smithi067.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-10T08:29:13.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:29:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:29:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:13 smithi134 bash[18722]: audit 2024-02-10T08:29:12.368058+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:13 smithi134 bash[18722]: audit 2024-02-10T08:29:13.163310+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? 172.21.15.67:0/2450805515' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi067-54245-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:13 smithi067 bash[17385]: audit 2024-02-10T08:29:12.368058+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:13 smithi067 bash[17385]: audit 2024-02-10T08:29:13.163310+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? 172.21.15.67:0/2450805515' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi067-54245-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:13 smithi067 bash[24488]: audit 2024-02-10T08:29:12.368058+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:13 smithi067 bash[24488]: audit 2024-02-10T08:29:13.163310+0000 mon.a (mon.0) 1057 : audit [INF] from='client.? 172.21.15.67:0/2450805515' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi067-54245-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:11.883202+0000 mgr.y (mgr.14547) 274 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: cluster 2024-02-10T08:29:12.915288+0000 mgr.y (mgr.14547) 275 : cluster [DBG] pgmap v239: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: cluster 2024-02-10T08:29:13.163890+0000 mon.a (mon.0) 1058 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.189719+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.369892+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.649208+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.704772+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.705164+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:29:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: cluster 2024-02-10T08:29:13.705314+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-10T08:29:14.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.711894+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? 172.21.15.67:0/1516450566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi067-54237-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:14.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.714179+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:14.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:14 smithi134 bash[18722]: audit 2024-02-10T08:29:13.718903+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:14.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:11.883202+0000 mgr.y (mgr.14547) 274 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:14.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: cluster 2024-02-10T08:29:12.915288+0000 mgr.y (mgr.14547) 275 : cluster [DBG] pgmap v239: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: cluster 2024-02-10T08:29:13.163890+0000 mon.a (mon.0) 1058 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.189719+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.369892+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.649208+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.704772+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.705164+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: cluster 2024-02-10T08:29:13.705314+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.711894+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? 172.21.15.67:0/1516450566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi067-54237-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.714179+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[24488]: audit 2024-02-10T08:29:13.718903+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:11.883202+0000 mgr.y (mgr.14547) 274 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:14.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: cluster 2024-02-10T08:29:12.915288+0000 mgr.y (mgr.14547) 275 : cluster [DBG] pgmap v239: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: cluster 2024-02-10T08:29:13.163890+0000 mon.a (mon.0) 1058 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.189719+0000 mon.c (mon.2) 230 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.369892+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.649208+0000 mon.a (mon.0) 1059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.704772+0000 mon.a (mon.0) 1060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi067-54461-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.705164+0000 mon.a (mon.0) 1061 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: cluster 2024-02-10T08:29:13.705314+0000 mon.a (mon.0) 1062 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.711894+0000 mon.a (mon.0) 1063 : audit [INF] from='client.? 172.21.15.67:0/1516450566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi067-54237-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.714179+0000 mon.c (mon.2) 232 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:14.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:14 smithi067 bash[17385]: audit 2024-02-10T08:29:13.718903+0000 mon.a (mon.0) 1064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]: dispatch 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: audit 2024-02-10T08:29:14.371567+0000 mon.c (mon.2) 233 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: cluster 2024-02-10T08:29:14.697033+0000 mon.a (mon.0) 1065 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: audit 2024-02-10T08:29:14.707285+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? 172.21.15.67:0/1516450566' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi067-54237-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: audit 2024-02-10T08:29:14.707785+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]': finished 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: cluster 2024-02-10T08:29:14.707946+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: audit 2024-02-10T08:29:14.715892+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.67:0/2423239097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:15 smithi134 bash[18722]: audit 2024-02-10T08:29:14.717476+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: audit 2024-02-10T08:29:14.371567+0000 mon.c (mon.2) 233 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: cluster 2024-02-10T08:29:14.697033+0000 mon.a (mon.0) 1065 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:29:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: audit 2024-02-10T08:29:14.707285+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? 172.21.15.67:0/1516450566' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi067-54237-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: audit 2024-02-10T08:29:14.707785+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]': finished 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: cluster 2024-02-10T08:29:14.707946+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: audit 2024-02-10T08:29:14.715892+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.67:0/2423239097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[17385]: audit 2024-02-10T08:29:14.717476+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: audit 2024-02-10T08:29:14.371567+0000 mon.c (mon.2) 233 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: cluster 2024-02-10T08:29:14.697033+0000 mon.a (mon.0) 1065 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: audit 2024-02-10T08:29:14.707285+0000 mon.a (mon.0) 1066 : audit [INF] from='client.? 172.21.15.67:0/1516450566' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi067-54237-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: audit 2024-02-10T08:29:14.707785+0000 mon.a (mon.0) 1067 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-11"}]': finished 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: cluster 2024-02-10T08:29:14.707946+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-10T08:29:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: audit 2024-02-10T08:29:14.715892+0000 mon.c (mon.2) 234 : audit [INF] from='client.? 172.21.15.67:0/2423239097' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:15.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:15 smithi067 bash[24488]: audit 2024-02-10T08:29:14.717476+0000 mon.a (mon.0) 1069 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:17.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:16 smithi134 bash[18722]: cluster 2024-02-10T08:29:14.916260+0000 mgr.y (mgr.14547) 276 : cluster [DBG] pgmap v242: 396 pgs: 72 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-02-10T08:29:17.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:16 smithi134 bash[18722]: audit 2024-02-10T08:29:15.372903+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:17.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:16 smithi134 bash[18722]: audit 2024-02-10T08:29:15.689905+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:17.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:16 smithi134 bash[18722]: cluster 2024-02-10T08:29:15.690150+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-10T08:29:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[17385]: cluster 2024-02-10T08:29:14.916260+0000 mgr.y (mgr.14547) 276 : cluster [DBG] pgmap v242: 396 pgs: 72 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-02-10T08:29:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[17385]: audit 2024-02-10T08:29:15.372903+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[17385]: audit 2024-02-10T08:29:15.689905+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[17385]: cluster 2024-02-10T08:29:15.690150+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-10T08:29:17.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[24488]: cluster 2024-02-10T08:29:14.916260+0000 mgr.y (mgr.14547) 276 : cluster [DBG] pgmap v242: 396 pgs: 72 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-02-10T08:29:17.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[24488]: audit 2024-02-10T08:29:15.372903+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:17.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[24488]: audit 2024-02-10T08:29:15.689905+0000 mon.a (mon.0) 1070 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi067-54245-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:17.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:16 smithi067 bash[24488]: cluster 2024-02-10T08:29:15.690150+0000 mon.a (mon.0) 1071 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-10T08:29:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:17 smithi134 bash[18722]: audit 2024-02-10T08:29:16.374708+0000 mon.c (mon.2) 236 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:17 smithi134 bash[18722]: cluster 2024-02-10T08:29:16.719204+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-10T08:29:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:17 smithi134 bash[18722]: audit 2024-02-10T08:29:16.730766+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:17 smithi134 bash[18722]: audit 2024-02-10T08:29:16.741176+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:17 smithi134 bash[18722]: audit 2024-02-10T08:29:16.742033+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? 172.21.15.67:0/604663677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi067-54237-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:17 smithi134 bash[18722]: audit 2024-02-10T08:29:17.376305+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:17] "GET /metrics HTTP/1.1" 200 193694 "" "Prometheus/2.33.4" 2024-02-10T08:29:18.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[24488]: audit 2024-02-10T08:29:16.374708+0000 mon.c (mon.2) 236 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[24488]: cluster 2024-02-10T08:29:16.719204+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[24488]: audit 2024-02-10T08:29:16.730766+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[24488]: audit 2024-02-10T08:29:16.741176+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[24488]: audit 2024-02-10T08:29:16.742033+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? 172.21.15.67:0/604663677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi067-54237-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[24488]: audit 2024-02-10T08:29:17.376305+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17385]: audit 2024-02-10T08:29:16.374708+0000 mon.c (mon.2) 236 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17385]: cluster 2024-02-10T08:29:16.719204+0000 mon.a (mon.0) 1072 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17385]: audit 2024-02-10T08:29:16.730766+0000 mon.c (mon.2) 237 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17385]: audit 2024-02-10T08:29:16.741176+0000 mon.a (mon.0) 1073 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17385]: audit 2024-02-10T08:29:16.742033+0000 mon.a (mon.0) 1074 : audit [INF] from='client.? 172.21.15.67:0/604663677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi067-54237-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:17 smithi067 bash[17385]: audit 2024-02-10T08:29:17.376305+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: cluster 2024-02-10T08:29:16.917185+0000 mgr.y (mgr.14547) 277 : cluster [DBG] pgmap v245: 364 pgs: 64 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: cluster 2024-02-10T08:29:17.713211+0000 mon.a (mon.0) 1075 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: audit 2024-02-10T08:29:17.728336+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: audit 2024-02-10T08:29:17.728542+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? 172.21.15.67:0/604663677' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi067-54237-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: cluster 2024-02-10T08:29:17.728875+0000 mon.a (mon.0) 1078 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: audit 2024-02-10T08:29:17.733311+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.67:0/3122848011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: audit 2024-02-10T08:29:17.740545+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:19.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:18 smithi134 bash[18722]: audit 2024-02-10T08:29:18.378142+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:19.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: cluster 2024-02-10T08:29:16.917185+0000 mgr.y (mgr.14547) 277 : cluster [DBG] pgmap v245: 364 pgs: 64 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:19.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: cluster 2024-02-10T08:29:17.713211+0000 mon.a (mon.0) 1075 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: audit 2024-02-10T08:29:17.728336+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: audit 2024-02-10T08:29:17.728542+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? 172.21.15.67:0/604663677' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi067-54237-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: cluster 2024-02-10T08:29:17.728875+0000 mon.a (mon.0) 1078 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-10T08:29:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: audit 2024-02-10T08:29:17.733311+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.67:0/3122848011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: audit 2024-02-10T08:29:17.740545+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[17385]: audit 2024-02-10T08:29:18.378142+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: cluster 2024-02-10T08:29:16.917185+0000 mgr.y (mgr.14547) 277 : cluster [DBG] pgmap v245: 364 pgs: 64 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: cluster 2024-02-10T08:29:17.713211+0000 mon.a (mon.0) 1075 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: audit 2024-02-10T08:29:17.728336+0000 mon.a (mon.0) 1076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: audit 2024-02-10T08:29:17.728542+0000 mon.a (mon.0) 1077 : audit [INF] from='client.? 172.21.15.67:0/604663677' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi067-54237-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: cluster 2024-02-10T08:29:17.728875+0000 mon.a (mon.0) 1078 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: audit 2024-02-10T08:29:17.733311+0000 mon.b (mon.1) 59 : audit [INF] from='client.? 172.21.15.67:0/3122848011' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: audit 2024-02-10T08:29:17.740545+0000 mon.a (mon.0) 1079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:19.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:18 smithi067 bash[24488]: audit 2024-02-10T08:29:18.378142+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:20.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:19 smithi134 bash[18722]: audit 2024-02-10T08:29:18.736606+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:20.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:19 smithi134 bash[18722]: cluster 2024-02-10T08:29:18.737233+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-10T08:29:20.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:19 smithi134 bash[18722]: cluster 2024-02-10T08:29:18.918099+0000 mgr.y (mgr.14547) 278 : cluster [DBG] pgmap v248: 364 pgs: 64 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:20.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:19 smithi134 bash[18722]: audit 2024-02-10T08:29:19.379913+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[24488]: audit 2024-02-10T08:29:18.736606+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[24488]: cluster 2024-02-10T08:29:18.737233+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-10T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[24488]: cluster 2024-02-10T08:29:18.918099+0000 mgr.y (mgr.14547) 278 : cluster [DBG] pgmap v248: 364 pgs: 64 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[24488]: audit 2024-02-10T08:29:19.379913+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:20.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[17385]: audit 2024-02-10T08:29:18.736606+0000 mon.a (mon.0) 1080 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi067-54245-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:20.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[17385]: cluster 2024-02-10T08:29:18.737233+0000 mon.a (mon.0) 1081 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-10T08:29:20.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[17385]: cluster 2024-02-10T08:29:18.918099+0000 mgr.y (mgr.14547) 278 : cluster [DBG] pgmap v248: 364 pgs: 64 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:20.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:19 smithi067 bash[17385]: audit 2024-02-10T08:29:19.379913+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: cluster 2024-02-10T08:29:19.739276+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: audit 2024-02-10T08:29:19.743293+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.67:0/3015745907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: audit 2024-02-10T08:29:19.747465+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: audit 2024-02-10T08:29:20.381656+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: audit 2024-02-10T08:29:20.696556+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: cluster 2024-02-10T08:29:20.696973+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: audit 2024-02-10T08:29:20.701907+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.67:0/2375770371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:20 smithi134 bash[18722]: audit 2024-02-10T08:29:20.714990+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: cluster 2024-02-10T08:29:19.739276+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-10T08:29:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: audit 2024-02-10T08:29:19.743293+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.67:0/3015745907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: audit 2024-02-10T08:29:19.747465+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: audit 2024-02-10T08:29:20.381656+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: audit 2024-02-10T08:29:20.696556+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: cluster 2024-02-10T08:29:20.696973+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: audit 2024-02-10T08:29:20.701907+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.67:0/2375770371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[17385]: audit 2024-02-10T08:29:20.714990+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: cluster 2024-02-10T08:29:19.739276+0000 mon.a (mon.0) 1082 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: audit 2024-02-10T08:29:19.743293+0000 mon.c (mon.2) 241 : audit [INF] from='client.? 172.21.15.67:0/3015745907' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: audit 2024-02-10T08:29:19.747465+0000 mon.a (mon.0) 1083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: audit 2024-02-10T08:29:20.381656+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: audit 2024-02-10T08:29:20.696556+0000 mon.a (mon.0) 1084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi067-54237-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:21.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: cluster 2024-02-10T08:29:20.696973+0000 mon.a (mon.0) 1085 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-10T08:29:21.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: audit 2024-02-10T08:29:20.701907+0000 mon.b (mon.1) 60 : audit [INF] from='client.? 172.21.15.67:0/2375770371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:21.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:20 smithi067 bash[24488]: audit 2024-02-10T08:29:20.714990+0000 mon.a (mon.0) 1086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:22.232 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:29:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:29:22.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:22 smithi134 bash[18722]: cluster 2024-02-10T08:29:20.919034+0000 mgr.y (mgr.14547) 279 : cluster [DBG] pgmap v251: 396 pgs: 96 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:22 smithi134 bash[18722]: audit 2024-02-10T08:29:21.383354+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:22 smithi134 bash[18722]: audit 2024-02-10T08:29:21.700805+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:22 smithi134 bash[18722]: cluster 2024-02-10T08:29:21.701169+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-10T08:29:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[17385]: cluster 2024-02-10T08:29:20.919034+0000 mgr.y (mgr.14547) 279 : cluster [DBG] pgmap v251: 396 pgs: 96 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[17385]: audit 2024-02-10T08:29:21.383354+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[17385]: audit 2024-02-10T08:29:21.700805+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[17385]: cluster 2024-02-10T08:29:21.701169+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-10T08:29:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[24488]: cluster 2024-02-10T08:29:20.919034+0000 mgr.y (mgr.14547) 279 : cluster [DBG] pgmap v251: 396 pgs: 96 unknown, 300 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[24488]: audit 2024-02-10T08:29:21.383354+0000 mon.c (mon.2) 243 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[24488]: audit 2024-02-10T08:29:21.700805+0000 mon.a (mon.0) 1087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi067-54245-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:22 smithi067 bash[24488]: cluster 2024-02-10T08:29:21.701169+0000 mon.a (mon.0) 1088 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:21.892476+0000 mgr.y (mgr.14547) 280 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:21.954029+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.220683+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.385097+0000 mon.c (mon.2) 245 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.705320+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: cluster 2024-02-10T08:29:22.705521+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.709983+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.710996+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.714812+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.67:0/3074013615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.719538+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:29:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.719991+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:23.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:23 smithi134 bash[18722]: audit 2024-02-10T08:29:22.728472+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:21.892476+0000 mgr.y (mgr.14547) 280 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:21.954029+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.220683+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.385097+0000 mon.c (mon.2) 245 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.705320+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: cluster 2024-02-10T08:29:22.705521+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.709983+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.710996+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.714812+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.67:0/3074013615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.719538+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:29:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.719991+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[17385]: audit 2024-02-10T08:29:22.728472+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:21.892476+0000 mgr.y (mgr.14547) 280 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:21.954029+0000 mon.c (mon.2) 244 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.220683+0000 mon.a (mon.0) 1089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.385097+0000 mon.c (mon.2) 245 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.705320+0000 mon.a (mon.0) 1090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: cluster 2024-02-10T08:29:22.705521+0000 mon.a (mon.0) 1091 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.709983+0000 mon.c (mon.2) 246 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.710996+0000 mon.c (mon.2) 247 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.714812+0000 mon.c (mon.2) 248 : audit [INF] from='client.? 172.21.15.67:0/3074013615' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.719538+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:29:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.719991+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:23.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:23 smithi067 bash[24488]: audit 2024-02-10T08:29:22.728472+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: cluster 2024-02-10T08:29:22.919980+0000 mgr.y (mgr.14547) 281 : cluster [DBG] pgmap v254: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 96 KiB/s wr, 100 op/s 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.386876+0000 mon.c (mon.2) 249 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: cluster 2024-02-10T08:29:23.700098+0000 mon.a (mon.0) 1095 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.708694+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]': finished 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.709011+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.709255+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: cluster 2024-02-10T08:29:23.709623+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-10T08:29:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.711589+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]: dispatch 2024-02-10T08:29:24.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.712062+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:24.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.713361+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.67:0/3607059083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.722211+0000 mon.a (mon.0) 1100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]: dispatch 2024-02-10T08:29:24.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.722834+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:24.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:24 smithi134 bash[18722]: audit 2024-02-10T08:29:23.725465+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: cluster 2024-02-10T08:29:22.919980+0000 mgr.y (mgr.14547) 281 : cluster [DBG] pgmap v254: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 96 KiB/s wr, 100 op/s 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.386876+0000 mon.c (mon.2) 249 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: cluster 2024-02-10T08:29:23.700098+0000 mon.a (mon.0) 1095 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.708694+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]': finished 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.709011+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.709255+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: cluster 2024-02-10T08:29:23.709623+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.711589+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]: dispatch 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.712062+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.713361+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.67:0/3607059083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.722211+0000 mon.a (mon.0) 1100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]: dispatch 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.722834+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[17385]: audit 2024-02-10T08:29:23.725465+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: cluster 2024-02-10T08:29:22.919980+0000 mgr.y (mgr.14547) 281 : cluster [DBG] pgmap v254: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 96 KiB/s wr, 100 op/s 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.386876+0000 mon.c (mon.2) 249 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: cluster 2024-02-10T08:29:23.700098+0000 mon.a (mon.0) 1095 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.708694+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-13"}]': finished 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.709011+0000 mon.a (mon.0) 1097 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.709255+0000 mon.a (mon.0) 1098 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi067-54237-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: cluster 2024-02-10T08:29:23.709623+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.711589+0000 mon.c (mon.2) 250 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.712062+0000 mon.c (mon.2) 251 : audit [INF] from='client.? 172.21.15.67:0/234205946' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.713361+0000 mon.c (mon.2) 252 : audit [INF] from='client.? 172.21.15.67:0/3607059083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.722211+0000 mon.a (mon.0) 1100 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.722834+0000 mon.a (mon.0) 1101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]: dispatch 2024-02-10T08:29:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:24 smithi067 bash[24488]: audit 2024-02-10T08:29:23.725465+0000 mon.a (mon.0) 1102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:24.720 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (4660 ms) 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2926 ms) 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3909 ms) 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2425 ms) 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (13920 ms total) 2024-02-10T08:29:24.721 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4420 ms) 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4258 ms) 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (8678 ms total) 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2687 ms) 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2210 ms) 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2336 ms) 2024-02-10T08:29:24.722 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1977 ms) 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9210 ms total) 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4059 ms) 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3972 ms) 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8031 ms total) 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (67867 ms total) 2024-02-10T08:29:24.723 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-02-10T08:29:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:25 smithi134 bash[18722]: audit 2024-02-10T08:29:24.388497+0000 mon.c (mon.2) 253 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:25 smithi134 bash[18722]: cluster 2024-02-10T08:29:24.703544+0000 mon.a (mon.0) 1103 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:25 smithi134 bash[18722]: audit 2024-02-10T08:29:24.712485+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]': finished 2024-02-10T08:29:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:25 smithi134 bash[18722]: audit 2024-02-10T08:29:24.712740+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:25 smithi134 bash[18722]: audit 2024-02-10T08:29:24.712937+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:25 smithi134 bash[18722]: cluster 2024-02-10T08:29:24.713417+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-10T08:29:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[17385]: audit 2024-02-10T08:29:24.388497+0000 mon.c (mon.2) 253 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[17385]: cluster 2024-02-10T08:29:24.703544+0000 mon.a (mon.0) 1103 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[17385]: audit 2024-02-10T08:29:24.712485+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]': finished 2024-02-10T08:29:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[17385]: audit 2024-02-10T08:29:24.712740+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[17385]: audit 2024-02-10T08:29:24.712937+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[17385]: cluster 2024-02-10T08:29:24.713417+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-10T08:29:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[24488]: audit 2024-02-10T08:29:24.388497+0000 mon.c (mon.2) 253 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[24488]: cluster 2024-02-10T08:29:24.703544+0000 mon.a (mon.0) 1103 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:29:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[24488]: audit 2024-02-10T08:29:24.712485+0000 mon.a (mon.0) 1104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-13", "mode": "writeback"}]': finished 2024-02-10T08:29:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[24488]: audit 2024-02-10T08:29:24.712740+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi067-54461-14"}]': finished 2024-02-10T08:29:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[24488]: audit 2024-02-10T08:29:24.712937+0000 mon.a (mon.0) 1106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi067-54245-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:25 smithi067 bash[24488]: cluster 2024-02-10T08:29:24.713417+0000 mon.a (mon.0) 1107 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: cluster 2024-02-10T08:29:24.921243+0000 mgr.y (mgr.14547) 282 : cluster [DBG] pgmap v257: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 96 KiB/s wr, 100 op/s 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.390318+0000 mon.c (mon.2) 254 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: cluster 2024-02-10T08:29:25.719050+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.722494+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.67:0/3003308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.724669+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.809494+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.810595+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-02-10T08:29:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.811489+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.812313+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.813148+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.814064+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.814877+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.815708+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.816554+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-02-10T08:29:26.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:26 smithi134 bash[18722]: audit 2024-02-10T08:29:25.817371+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-02-10T08:29:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: cluster 2024-02-10T08:29:24.921243+0000 mgr.y (mgr.14547) 282 : cluster [DBG] pgmap v257: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 96 KiB/s wr, 100 op/s 2024-02-10T08:29:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.390318+0000 mon.c (mon.2) 254 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: cluster 2024-02-10T08:29:25.719050+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.722494+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.67:0/3003308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.724669+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.809494+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.810595+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.811489+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.812313+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.813148+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.814064+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.814877+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.815708+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.816554+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-02-10T08:29:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[17385]: audit 2024-02-10T08:29:25.817371+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: cluster 2024-02-10T08:29:24.921243+0000 mgr.y (mgr.14547) 282 : cluster [DBG] pgmap v257: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 96 KiB/s wr, 100 op/s 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.390318+0000 mon.c (mon.2) 254 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: cluster 2024-02-10T08:29:25.719050+0000 mon.a (mon.0) 1108 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.722494+0000 mon.b (mon.1) 61 : audit [INF] from='client.? 172.21.15.67:0/3003308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.724669+0000 mon.a (mon.0) 1109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.809494+0000 mon.c (mon.2) 255 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.810595+0000 mon.c (mon.2) 256 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.811489+0000 mon.c (mon.2) 257 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.812313+0000 mon.c (mon.2) 258 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.813148+0000 mon.c (mon.2) 259 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.814064+0000 mon.c (mon.2) 260 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-02-10T08:29:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.814877+0000 mon.c (mon.2) 261 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-02-10T08:29:26.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.815708+0000 mon.c (mon.2) 262 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-02-10T08:29:26.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.816554+0000 mon.c (mon.2) 263 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-02-10T08:29:26.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:26 smithi067 bash[24488]: audit 2024-02-10T08:29:25.817371+0000 mon.c (mon.2) 264 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.809818+0000 mgr.y (mgr.14547) 283 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.810824+0000 mgr.y (mgr.14547) 284 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.811683+0000 mgr.y (mgr.14547) 285 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.812512+0000 mgr.y (mgr.14547) 286 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.813355+0000 mgr.y (mgr.14547) 287 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.814272+0000 mgr.y (mgr.14547) 288 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.815082+0000 mgr.y (mgr.14547) 289 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-02-10T08:29:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.815920+0000 mgr.y (mgr.14547) 290 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.816755+0000 mgr.y (mgr.14547) 291 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:25.817590+0000 mgr.y (mgr.14547) 292 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: cluster 2024-02-10T08:29:25.931948+0000 osd.1 (osd.1) 7 : cluster [DBG] 96.3 scrub starts 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: cluster 2024-02-10T08:29:25.935490+0000 osd.1 (osd.1) 8 : cluster [DBG] 96.3 scrub ok 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:26.391988+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.680164+0000 osd.5 (osd.5) 7 : cluster [DBG] 96.7 scrub starts 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.683421+0000 osd.5 (osd.5) 8 : cluster [DBG] 96.7 scrub ok 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:26.726025+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.726974+0000 mon.a (mon.0) 1111 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-10T08:29:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:27 smithi134 bash[18722]: audit 2024-02-10T08:29:26.731499+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.67:0/887145551' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi067-54245-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.809818+0000 mgr.y (mgr.14547) 283 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-02-10T08:29:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.810824+0000 mgr.y (mgr.14547) 284 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-02-10T08:29:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.811683+0000 mgr.y (mgr.14547) 285 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.812512+0000 mgr.y (mgr.14547) 286 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.813355+0000 mgr.y (mgr.14547) 287 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.814272+0000 mgr.y (mgr.14547) 288 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.815082+0000 mgr.y (mgr.14547) 289 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.815920+0000 mgr.y (mgr.14547) 290 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.816755+0000 mgr.y (mgr.14547) 291 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:25.817590+0000 mgr.y (mgr.14547) 292 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: cluster 2024-02-10T08:29:25.931948+0000 osd.1 (osd.1) 7 : cluster [DBG] 96.3 scrub starts 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: cluster 2024-02-10T08:29:25.935490+0000 osd.1 (osd.1) 8 : cluster [DBG] 96.3 scrub ok 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:26.391988+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.680164+0000 osd.5 (osd.5) 7 : cluster [DBG] 96.7 scrub starts 2024-02-10T08:29:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.683421+0000 osd.5 (osd.5) 8 : cluster [DBG] 96.7 scrub ok 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:26.726025+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.726974+0000 mon.a (mon.0) 1111 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17385]: audit 2024-02-10T08:29:26.731499+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.67:0/887145551' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi067-54245-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.809818+0000 mgr.y (mgr.14547) 283 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.0"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.810824+0000 mgr.y (mgr.14547) 284 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.1"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.811683+0000 mgr.y (mgr.14547) 285 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.2"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.812512+0000 mgr.y (mgr.14547) 286 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.3"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.813355+0000 mgr.y (mgr.14547) 287 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.4"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.814272+0000 mgr.y (mgr.14547) 288 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.5"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.815082+0000 mgr.y (mgr.14547) 289 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.6"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.815920+0000 mgr.y (mgr.14547) 290 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.7"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.816755+0000 mgr.y (mgr.14547) 291 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.8"}]: dispatch 2024-02-10T08:29:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:25.817590+0000 mgr.y (mgr.14547) 292 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "96.9"}]: dispatch 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: cluster 2024-02-10T08:29:25.931948+0000 osd.1 (osd.1) 7 : cluster [DBG] 96.3 scrub starts 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: cluster 2024-02-10T08:29:25.935490+0000 osd.1 (osd.1) 8 : cluster [DBG] 96.3 scrub ok 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:26.391988+0000 mon.c (mon.2) 265 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.680164+0000 osd.5 (osd.5) 7 : cluster [DBG] 96.7 scrub starts 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.683421+0000 osd.5 (osd.5) 8 : cluster [DBG] 96.7 scrub ok 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:26.726025+0000 mon.a (mon.0) 1110 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi067-54237-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.726974+0000 mon.a (mon.0) 1111 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-10T08:29:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[24488]: audit 2024-02-10T08:29:26.731499+0000 mon.a (mon.0) 1112 : audit [INF] from='client.? 172.21.15.67:0/887145551' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi067-54245-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:29:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:27] "GET /metrics HTTP/1.1" 200 193694 "" "Prometheus/2.33.4" 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.866322+0000 osd.6 (osd.6) 7 : cluster [DBG] 96.8 scrub starts 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.871921+0000 osd.6 (osd.6) 8 : cluster [DBG] 96.8 scrub ok 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.920443+0000 osd.3 (osd.3) 11 : cluster [DBG] 96.6 scrub starts 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.922470+0000 mgr.y (mgr.14547) 293 : cluster [DBG] pgmap v260: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 25 KiB/s wr, 101 op/s 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: cluster 2024-02-10T08:29:26.926705+0000 osd.3 (osd.3) 12 : cluster [DBG] 96.6 scrub ok 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: audit 2024-02-10T08:29:27.393910+0000 mon.c (mon.2) 266 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: audit 2024-02-10T08:29:27.735081+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.67:0/887145551' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi067-54245-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:28.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:28 smithi134 bash[18722]: cluster 2024-02-10T08:29:27.735307+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-10T08:29:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.866322+0000 osd.6 (osd.6) 7 : cluster [DBG] 96.8 scrub starts 2024-02-10T08:29:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.871921+0000 osd.6 (osd.6) 8 : cluster [DBG] 96.8 scrub ok 2024-02-10T08:29:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.920443+0000 osd.3 (osd.3) 11 : cluster [DBG] 96.6 scrub starts 2024-02-10T08:29:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.922470+0000 mgr.y (mgr.14547) 293 : cluster [DBG] pgmap v260: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 25 KiB/s wr, 101 op/s 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: cluster 2024-02-10T08:29:26.926705+0000 osd.3 (osd.3) 12 : cluster [DBG] 96.6 scrub ok 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: audit 2024-02-10T08:29:27.393910+0000 mon.c (mon.2) 266 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: audit 2024-02-10T08:29:27.735081+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.67:0/887145551' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi067-54245-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[17385]: cluster 2024-02-10T08:29:27.735307+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.866322+0000 osd.6 (osd.6) 7 : cluster [DBG] 96.8 scrub starts 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.871921+0000 osd.6 (osd.6) 8 : cluster [DBG] 96.8 scrub ok 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.920443+0000 osd.3 (osd.3) 11 : cluster [DBG] 96.6 scrub starts 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.922470+0000 mgr.y (mgr.14547) 293 : cluster [DBG] pgmap v260: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 25 KiB/s wr, 101 op/s 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: cluster 2024-02-10T08:29:26.926705+0000 osd.3 (osd.3) 12 : cluster [DBG] 96.6 scrub ok 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: audit 2024-02-10T08:29:27.393910+0000 mon.c (mon.2) 266 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: audit 2024-02-10T08:29:27.735081+0000 mon.a (mon.0) 1113 : audit [INF] from='client.? 172.21.15.67:0/887145551' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi067-54245-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:28 smithi067 bash[24488]: cluster 2024-02-10T08:29:27.735307+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-10T08:29:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:29 smithi134 bash[18722]: audit 2024-02-10T08:29:28.395793+0000 mon.c (mon.2) 267 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:29 smithi134 bash[18722]: cluster 2024-02-10T08:29:28.757117+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-10T08:29:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:29 smithi134 bash[18722]: audit 2024-02-10T08:29:28.764427+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.67:0/2749071315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:29 smithi134 bash[18722]: audit 2024-02-10T08:29:28.771323+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[24488]: audit 2024-02-10T08:29:28.395793+0000 mon.c (mon.2) 267 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[24488]: cluster 2024-02-10T08:29:28.757117+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-10T08:29:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[24488]: audit 2024-02-10T08:29:28.764427+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.67:0/2749071315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[24488]: audit 2024-02-10T08:29:28.771323+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:30.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[17385]: audit 2024-02-10T08:29:28.395793+0000 mon.c (mon.2) 267 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:30.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[17385]: cluster 2024-02-10T08:29:28.757117+0000 mon.a (mon.0) 1115 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-10T08:29:30.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[17385]: audit 2024-02-10T08:29:28.764427+0000 mon.b (mon.1) 62 : audit [INF] from='client.? 172.21.15.67:0/2749071315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:30.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:29 smithi067 bash[17385]: audit 2024-02-10T08:29:28.771323+0000 mon.a (mon.0) 1116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:28.923396+0000 mgr.y (mgr.14547) 294 : cluster [DBG] pgmap v263: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 91 op/s 2024-02-10T08:29:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:28.935355+0000 osd.2 (osd.2) 7 : cluster [DBG] 96.0 scrub starts 2024-02-10T08:29:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:28.939640+0000 osd.2 (osd.2) 8 : cluster [DBG] 96.0 scrub ok 2024-02-10T08:29:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: audit 2024-02-10T08:29:29.397557+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:29.748057+0000 osd.4 (osd.4) 7 : cluster [DBG] 96.1 scrub starts 2024-02-10T08:29:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:29.751417+0000 osd.4 (osd.4) 8 : cluster [DBG] 96.1 scrub ok 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: audit 2024-02-10T08:29:29.760169+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:29.760472+0000 mon.a (mon.0) 1118 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: audit 2024-02-10T08:29:29.764948+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.67:0/521017481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: audit 2024-02-10T08:29:29.768501+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.67:0/3624501988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi067-54245-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: audit 2024-02-10T08:29:29.787340+0000 mon.a (mon.0) 1120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:30.159206+0000 osd.7 (osd.7) 3 : cluster [DBG] 96.5 scrub starts 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: cluster 2024-02-10T08:29:30.163051+0000 osd.7 (osd.7) 4 : cluster [DBG] 96.5 scrub ok 2024-02-10T08:29:31.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:30 smithi134 bash[18722]: audit 2024-02-10T08:29:30.399261+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:28.923396+0000 mgr.y (mgr.14547) 294 : cluster [DBG] pgmap v263: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 91 op/s 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:28.935355+0000 osd.2 (osd.2) 7 : cluster [DBG] 96.0 scrub starts 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:28.939640+0000 osd.2 (osd.2) 8 : cluster [DBG] 96.0 scrub ok 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: audit 2024-02-10T08:29:29.397557+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:29.748057+0000 osd.4 (osd.4) 7 : cluster [DBG] 96.1 scrub starts 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:29.751417+0000 osd.4 (osd.4) 8 : cluster [DBG] 96.1 scrub ok 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: audit 2024-02-10T08:29:29.760169+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:29.760472+0000 mon.a (mon.0) 1118 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: audit 2024-02-10T08:29:29.764948+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.67:0/521017481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: audit 2024-02-10T08:29:29.768501+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.67:0/3624501988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi067-54245-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: audit 2024-02-10T08:29:29.787340+0000 mon.a (mon.0) 1120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:30.159206+0000 osd.7 (osd.7) 3 : cluster [DBG] 96.5 scrub starts 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: cluster 2024-02-10T08:29:30.163051+0000 osd.7 (osd.7) 4 : cluster [DBG] 96.5 scrub ok 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[17385]: audit 2024-02-10T08:29:30.399261+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:28.923396+0000 mgr.y (mgr.14547) 294 : cluster [DBG] pgmap v263: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 91 op/s 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:28.935355+0000 osd.2 (osd.2) 7 : cluster [DBG] 96.0 scrub starts 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:28.939640+0000 osd.2 (osd.2) 8 : cluster [DBG] 96.0 scrub ok 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: audit 2024-02-10T08:29:29.397557+0000 mon.c (mon.2) 268 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:29.748057+0000 osd.4 (osd.4) 7 : cluster [DBG] 96.1 scrub starts 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:29.751417+0000 osd.4 (osd.4) 8 : cluster [DBG] 96.1 scrub ok 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: audit 2024-02-10T08:29:29.760169+0000 mon.a (mon.0) 1117 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi067-54237-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:29.760472+0000 mon.a (mon.0) 1118 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-10T08:29:31.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: audit 2024-02-10T08:29:29.764948+0000 mon.c (mon.2) 269 : audit [INF] from='client.? 172.21.15.67:0/521017481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: audit 2024-02-10T08:29:29.768501+0000 mon.a (mon.0) 1119 : audit [INF] from='client.? 172.21.15.67:0/3624501988' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi067-54245-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: audit 2024-02-10T08:29:29.787340+0000 mon.a (mon.0) 1120 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:31.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:30.159206+0000 osd.7 (osd.7) 3 : cluster [DBG] 96.5 scrub starts 2024-02-10T08:29:31.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: cluster 2024-02-10T08:29:30.163051+0000 osd.7 (osd.7) 4 : cluster [DBG] 96.5 scrub ok 2024-02-10T08:29:31.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:30 smithi067 bash[24488]: audit 2024-02-10T08:29:30.399261+0000 mon.c (mon.2) 270 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:32.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: cluster 2024-02-10T08:29:29.978925+0000 osd.2 (osd.2) 9 : cluster [DBG] 96.9 scrub starts 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: cluster 2024-02-10T08:29:29.983706+0000 osd.2 (osd.2) 10 : cluster [DBG] 96.9 scrub ok 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: cluster 2024-02-10T08:29:30.717695+0000 osd.4 (osd.4) 9 : cluster [DBG] 96.2 scrub starts 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: cluster 2024-02-10T08:29:30.722476+0000 osd.4 (osd.4) 10 : cluster [DBG] 96.2 scrub ok 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: audit 2024-02-10T08:29:30.768717+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.67:0/3624501988' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi067-54245-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: audit 2024-02-10T08:29:30.769010+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: cluster 2024-02-10T08:29:30.769271+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: audit 2024-02-10T08:29:30.789913+0000 mon.a (mon.0) 1124 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: cluster 2024-02-10T08:29:30.924038+0000 mgr.y (mgr.14547) 295 : cluster [DBG] pgmap v266: 356 pgs: 64 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:31 smithi134 bash[18722]: audit 2024-02-10T08:29:31.401132+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: cluster 2024-02-10T08:29:29.978925+0000 osd.2 (osd.2) 9 : cluster [DBG] 96.9 scrub starts 2024-02-10T08:29:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: cluster 2024-02-10T08:29:29.983706+0000 osd.2 (osd.2) 10 : cluster [DBG] 96.9 scrub ok 2024-02-10T08:29:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: cluster 2024-02-10T08:29:30.717695+0000 osd.4 (osd.4) 9 : cluster [DBG] 96.2 scrub starts 2024-02-10T08:29:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: cluster 2024-02-10T08:29:30.722476+0000 osd.4 (osd.4) 10 : cluster [DBG] 96.2 scrub ok 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: audit 2024-02-10T08:29:30.768717+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.67:0/3624501988' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi067-54245-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: audit 2024-02-10T08:29:30.769010+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: cluster 2024-02-10T08:29:30.769271+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: audit 2024-02-10T08:29:30.789913+0000 mon.a (mon.0) 1124 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: cluster 2024-02-10T08:29:30.924038+0000 mgr.y (mgr.14547) 295 : cluster [DBG] pgmap v266: 356 pgs: 64 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[24488]: audit 2024-02-10T08:29:31.401132+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: cluster 2024-02-10T08:29:29.978925+0000 osd.2 (osd.2) 9 : cluster [DBG] 96.9 scrub starts 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: cluster 2024-02-10T08:29:29.983706+0000 osd.2 (osd.2) 10 : cluster [DBG] 96.9 scrub ok 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: cluster 2024-02-10T08:29:30.717695+0000 osd.4 (osd.4) 9 : cluster [DBG] 96.2 scrub starts 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: cluster 2024-02-10T08:29:30.722476+0000 osd.4 (osd.4) 10 : cluster [DBG] 96.2 scrub ok 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: audit 2024-02-10T08:29:30.768717+0000 mon.a (mon.0) 1121 : audit [INF] from='client.? 172.21.15.67:0/3624501988' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi067-54245-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: audit 2024-02-10T08:29:30.769010+0000 mon.a (mon.0) 1122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54485-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: cluster 2024-02-10T08:29:30.769271+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-10T08:29:32.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: audit 2024-02-10T08:29:30.789913+0000 mon.a (mon.0) 1124 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:32.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: cluster 2024-02-10T08:29:30.924038+0000 mgr.y (mgr.14547) 295 : cluster [DBG] pgmap v266: 356 pgs: 64 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:32.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:31 smithi067 bash[17385]: audit 2024-02-10T08:29:31.401132+0000 mon.c (mon.2) 271 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[17385]: cluster 2024-02-10T08:29:31.800547+0000 mon.a (mon.0) 1125 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-10T08:29:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[17385]: audit 2024-02-10T08:29:31.815660+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.67:0/468753158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[17385]: audit 2024-02-10T08:29:31.817123+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[17385]: audit 2024-02-10T08:29:31.899234+0000 mgr.y (mgr.14547) 296 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[17385]: audit 2024-02-10T08:29:32.402947+0000 mon.c (mon.2) 273 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[24488]: cluster 2024-02-10T08:29:31.800547+0000 mon.a (mon.0) 1125 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[24488]: audit 2024-02-10T08:29:31.815660+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.67:0/468753158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[24488]: audit 2024-02-10T08:29:31.817123+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[24488]: audit 2024-02-10T08:29:31.899234+0000 mgr.y (mgr.14547) 296 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:32 smithi067 bash[24488]: audit 2024-02-10T08:29:32.402947+0000 mon.c (mon.2) 273 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:32 smithi134 bash[18722]: cluster 2024-02-10T08:29:31.800547+0000 mon.a (mon.0) 1125 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-10T08:29:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:32 smithi134 bash[18722]: audit 2024-02-10T08:29:31.815660+0000 mon.c (mon.2) 272 : audit [INF] from='client.? 172.21.15.67:0/468753158' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:32 smithi134 bash[18722]: audit 2024-02-10T08:29:31.817123+0000 mon.a (mon.0) 1126 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:32 smithi134 bash[18722]: audit 2024-02-10T08:29:31.899234+0000 mgr.y (mgr.14547) 296 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:33.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:32 smithi134 bash[18722]: audit 2024-02-10T08:29:32.402947+0000 mon.c (mon.2) 273 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:32.814781+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: cluster 2024-02-10T08:29:32.815717+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:32.819145+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.67:0/1794619933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:32.834471+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: cluster 2024-02-10T08:29:32.925020+0000 mgr.y (mgr.14547) 297 : cluster [DBG] pgmap v269: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:33.073828+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:33.074653+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:33.075129+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:33.404725+0000 mon.c (mon.2) 277 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:33.810119+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: audit 2024-02-10T08:29:33.810607+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[17385]: cluster 2024-02-10T08:29:33.811032+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:32.814781+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: cluster 2024-02-10T08:29:32.815717+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-10T08:29:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:32.819145+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.67:0/1794619933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:32.834471+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: cluster 2024-02-10T08:29:32.925020+0000 mgr.y (mgr.14547) 297 : cluster [DBG] pgmap v269: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:33.073828+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:33.074653+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:33.075129+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:33.404725+0000 mon.c (mon.2) 277 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:33.810119+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: audit 2024-02-10T08:29:33.810607+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:33 smithi067 bash[24488]: cluster 2024-02-10T08:29:33.811032+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:32.814781+0000 mon.a (mon.0) 1127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54237-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: cluster 2024-02-10T08:29:32.815717+0000 mon.a (mon.0) 1128 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:32.819145+0000 mon.c (mon.2) 274 : audit [INF] from='client.? 172.21.15.67:0/1794619933' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:32.834471+0000 mon.a (mon.0) 1129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: cluster 2024-02-10T08:29:32.925020+0000 mgr.y (mgr.14547) 297 : cluster [DBG] pgmap v269: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:33.073828+0000 mon.c (mon.2) 275 : audit [DBG] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:33.074653+0000 mon.c (mon.2) 276 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:33.075129+0000 mon.a (mon.0) 1130 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:33.404725+0000 mon.c (mon.2) 277 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:33.810119+0000 mon.a (mon.0) 1131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: audit 2024-02-10T08:29:33.810607+0000 mon.a (mon.0) 1132 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:33 smithi134 bash[18722]: cluster 2024-02-10T08:29:33.811032+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-10T08:29:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:33.831515+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2"}]: dispatch 2024-02-10T08:29:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:33.849894+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:33.855599+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:35.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:34.329950+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:34.332001+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:34.333105+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[17385]: audit 2024-02-10T08:29:34.406286+0000 mon.c (mon.2) 280 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:33.831515+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2"}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:33.849894+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:33.855599+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:34.329950+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:34.332001+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:34.333105+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:34 smithi067 bash[24488]: audit 2024-02-10T08:29:34.406286+0000 mon.c (mon.2) 280 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:33.831515+0000 mon.c (mon.2) 278 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2"}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:33.849894+0000 mon.c (mon.2) 279 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:33.855599+0000 mon.a (mon.0) 1134 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:34.329950+0000 mon.a (mon.0) 1135 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:34.332001+0000 mon.a (mon.0) 1136 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:34.333105+0000 mon.a (mon.0) 1137 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:34 smithi134 bash[18722]: audit 2024-02-10T08:29:34.406286+0000 mon.c (mon.2) 280 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: cluster 2024-02-10T08:29:34.267212+0000 osd.0 (osd.0) 15 : cluster [DBG] 96.4 scrub starts 2024-02-10T08:29:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: cluster 2024-02-10T08:29:34.269662+0000 osd.0 (osd.0) 16 : cluster [DBG] 96.4 scrub ok 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.857737+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.857966+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: cluster 2024-02-10T08:29:34.858070+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.860372+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.67:0/1304077623' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.860734+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.861398+0000 mon.a (mon.0) 1141 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.861430+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.864243+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:34.869928+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi067-54485-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: cluster 2024-02-10T08:29:34.925953+0000 mgr.y (mgr.14547) 298 : cluster [DBG] pgmap v272: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[17385]: audit 2024-02-10T08:29:35.407765+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: cluster 2024-02-10T08:29:34.267212+0000 osd.0 (osd.0) 15 : cluster [DBG] 96.4 scrub starts 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: cluster 2024-02-10T08:29:34.269662+0000 osd.0 (osd.0) 16 : cluster [DBG] 96.4 scrub ok 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.857737+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.857966+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: cluster 2024-02-10T08:29:34.858070+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.860372+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.67:0/1304077623' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.860734+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.861398+0000 mon.a (mon.0) 1141 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.861430+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.864243+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:34.869928+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi067-54485-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: cluster 2024-02-10T08:29:34.925953+0000 mgr.y (mgr.14547) 298 : cluster [DBG] pgmap v272: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:36.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:35 smithi067 bash[24488]: audit 2024-02-10T08:29:35.407765+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: cluster 2024-02-10T08:29:34.267212+0000 osd.0 (osd.0) 15 : cluster [DBG] 96.4 scrub starts 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: cluster 2024-02-10T08:29:34.269662+0000 osd.0 (osd.0) 16 : cluster [DBG] 96.4 scrub ok 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.857737+0000 mon.a (mon.0) 1138 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi067-54314-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.857966+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: cluster 2024-02-10T08:29:34.858070+0000 mon.a (mon.0) 1140 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.860372+0000 mon.c (mon.2) 281 : audit [INF] from='client.? 172.21.15.67:0/1304077623' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.860734+0000 mon.c (mon.2) 282 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.861398+0000 mon.a (mon.0) 1141 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.861430+0000 mon.c (mon.2) 283 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.864243+0000 mon.a (mon.0) 1142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-10T08:29:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:34.869928+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi067-54485-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: cluster 2024-02-10T08:29:34.925953+0000 mgr.y (mgr.14547) 298 : cluster [DBG] pgmap v272: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:36.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:35 smithi134 bash[18722]: audit 2024-02-10T08:29:35.407765+0000 mon.c (mon.2) 284 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: audit 2024-02-10T08:29:35.871699+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: audit 2024-02-10T08:29:35.871834+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: audit 2024-02-10T08:29:35.890441+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: cluster 2024-02-10T08:29:36.322791+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: audit 2024-02-10T08:29:36.327226+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: audit 2024-02-10T08:29:36.328366+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? 172.21.15.67:0/2646609124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi067-54245-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:36 smithi134 bash[18722]: audit 2024-02-10T08:29:36.409446+0000 mon.c (mon.2) 286 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: audit 2024-02-10T08:29:35.871699+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: audit 2024-02-10T08:29:35.871834+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:29:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: audit 2024-02-10T08:29:35.890441+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:29:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: cluster 2024-02-10T08:29:36.322791+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-10T08:29:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: audit 2024-02-10T08:29:36.327226+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: audit 2024-02-10T08:29:36.328366+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? 172.21.15.67:0/2646609124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi067-54245-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[24488]: audit 2024-02-10T08:29:36.409446+0000 mon.c (mon.2) 286 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: audit 2024-02-10T08:29:35.871699+0000 mon.a (mon.0) 1144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi067-54237-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: audit 2024-02-10T08:29:35.871834+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: audit 2024-02-10T08:29:35.890441+0000 mon.c (mon.2) 285 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: cluster 2024-02-10T08:29:36.322791+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: audit 2024-02-10T08:29:36.327226+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: audit 2024-02-10T08:29:36.328366+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? 172.21.15.67:0/2646609124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi067-54245-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:36 smithi067 bash[17385]: audit 2024-02-10T08:29:36.409446+0000 mon.c (mon.2) 286 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:37] "GET /metrics HTTP/1.1" 200 194195 "" "Prometheus/2.33.4" 2024-02-10T08:29:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: audit 2024-02-10T08:29:36.875732+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi067-54485-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: audit 2024-02-10T08:29:36.875968+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: audit 2024-02-10T08:29:36.876137+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? 172.21.15.67:0/2646609124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi067-54245-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: cluster 2024-02-10T08:29:36.876488+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: audit 2024-02-10T08:29:36.878706+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: audit 2024-02-10T08:29:36.900811+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: cluster 2024-02-10T08:29:36.926514+0000 mgr.y (mgr.14547) 299 : cluster [DBG] pgmap v275: 332 pgs: 32 creating+peering, 8 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T08:29:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[24488]: audit 2024-02-10T08:29:37.411101+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: audit 2024-02-10T08:29:36.875732+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi067-54485-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: audit 2024-02-10T08:29:36.875968+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: audit 2024-02-10T08:29:36.876137+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? 172.21.15.67:0/2646609124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi067-54245-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: cluster 2024-02-10T08:29:36.876488+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: audit 2024-02-10T08:29:36.878706+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: audit 2024-02-10T08:29:36.900811+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:29:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: cluster 2024-02-10T08:29:36.926514+0000 mgr.y (mgr.14547) 299 : cluster [DBG] pgmap v275: 332 pgs: 32 creating+peering, 8 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T08:29:38.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:37 smithi067 bash[17385]: audit 2024-02-10T08:29:37.411101+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: audit 2024-02-10T08:29:36.875732+0000 mon.a (mon.0) 1149 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi067-54485-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: audit 2024-02-10T08:29:36.875968+0000 mon.a (mon.0) 1150 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: audit 2024-02-10T08:29:36.876137+0000 mon.a (mon.0) 1151 : audit [INF] from='client.? 172.21.15.67:0/2646609124' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi067-54245-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: cluster 2024-02-10T08:29:36.876488+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: audit 2024-02-10T08:29:36.878706+0000 mon.c (mon.2) 287 : audit [INF] from='client.? 172.21.15.67:0/2778988589' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: audit 2024-02-10T08:29:36.900811+0000 mon.a (mon.0) 1153 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]: dispatch 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: cluster 2024-02-10T08:29:36.926514+0000 mgr.y (mgr.14547) 299 : cluster [DBG] pgmap v275: 332 pgs: 32 creating+peering, 8 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-10T08:29:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:37 smithi134 bash[18722]: audit 2024-02-10T08:29:37.411101+0000 mon.c (mon.2) 288 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:38.898 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: seed 54314 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (493 ms) 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (74 ms) 2024-02-10T08:29:38.899 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (32 ms) 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (9 ms) 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (8 ms) 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-10T08:29:38.900 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (11 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (4 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (2575 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (485 ms) 2024-02-10T08:29:38.901 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: done waiting, doing copies 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: done waiting 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60801 ms) 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (8 ms) 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (6 ms) 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4828 ms) 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-10T08:29:38.902 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-10T08:29:38.903 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: audit 2024-02-10T08:29:37.878978+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: cluster 2024-02-10T08:29:37.879261+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: audit 2024-02-10T08:29:37.883742+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.67:0/109861606' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: audit 2024-02-10T08:29:37.886962+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: audit 2024-02-10T08:29:38.412647+0000 mon.c (mon.2) 289 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: audit 2024-02-10T08:29:38.887755+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: cluster 2024-02-10T08:29:38.888169+0000 mon.a (mon.0) 1158 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-10T08:29:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:38 smithi134 bash[18722]: audit 2024-02-10T08:29:38.892743+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? 172.21.15.67:0/3366967636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi067-54245-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: audit 2024-02-10T08:29:37.878978+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: cluster 2024-02-10T08:29:37.879261+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: audit 2024-02-10T08:29:37.883742+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.67:0/109861606' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: audit 2024-02-10T08:29:37.886962+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: audit 2024-02-10T08:29:38.412647+0000 mon.c (mon.2) 289 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: audit 2024-02-10T08:29:38.887755+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: cluster 2024-02-10T08:29:38.888169+0000 mon.a (mon.0) 1158 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[17385]: audit 2024-02-10T08:29:38.892743+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? 172.21.15.67:0/3366967636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi067-54245-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: audit 2024-02-10T08:29:37.878978+0000 mon.a (mon.0) 1154 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi067-54314-1","app":"app1","key":"key1"}]': finished 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: cluster 2024-02-10T08:29:37.879261+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: audit 2024-02-10T08:29:37.883742+0000 mon.b (mon.1) 63 : audit [INF] from='client.? 172.21.15.67:0/109861606' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: audit 2024-02-10T08:29:37.886962+0000 mon.a (mon.0) 1156 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: audit 2024-02-10T08:29:38.412647+0000 mon.c (mon.2) 289 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:39.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: audit 2024-02-10T08:29:38.887755+0000 mon.a (mon.0) 1157 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi067-54237-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:39.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: cluster 2024-02-10T08:29:38.888169+0000 mon.a (mon.0) 1158 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-10T08:29:39.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:38 smithi067 bash[24488]: audit 2024-02-10T08:29:38.892743+0000 mon.a (mon.0) 1159 : audit [INF] from='client.? 172.21.15.67:0/3366967636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi067-54245-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: cluster 2024-02-10T08:29:38.927171+0000 mgr.y (mgr.14547) 300 : cluster [DBG] pgmap v278: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:38.939798+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:38.940242+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:38.941081+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:38.941388+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:38.941911+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:38.942228+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:39.414370+0000 mon.c (mon.2) 293 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:39.542449+0000 mon.c (mon.2) 294 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:39.886881+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? 172.21.15.67:0/3366967636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi067-54245-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:39.887426+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: cluster 2024-02-10T08:29:39.887613+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:39.892091+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:39 smithi134 bash[18722]: audit 2024-02-10T08:29:39.893060+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: cluster 2024-02-10T08:29:38.927171+0000 mgr.y (mgr.14547) 300 : cluster [DBG] pgmap v278: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:38.939798+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:38.940242+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:38.941081+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:38.941388+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:38.941911+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:38.942228+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:39.414370+0000 mon.c (mon.2) 293 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:39.542449+0000 mon.c (mon.2) 294 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:39.886881+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? 172.21.15.67:0/3366967636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi067-54245-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:39.887426+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: cluster 2024-02-10T08:29:39.887613+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:39.892091+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[17385]: audit 2024-02-10T08:29:39.893060+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: cluster 2024-02-10T08:29:38.927171+0000 mgr.y (mgr.14547) 300 : cluster [DBG] pgmap v278: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:38.939798+0000 mon.c (mon.2) 290 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:38.940242+0000 mon.a (mon.0) 1160 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:38.941081+0000 mon.c (mon.2) 291 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:38.941388+0000 mon.a (mon.0) 1161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:38.941911+0000 mon.c (mon.2) 292 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:38.942228+0000 mon.a (mon.0) 1162 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:39.414370+0000 mon.c (mon.2) 293 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:39.542449+0000 mon.c (mon.2) 294 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:39.886881+0000 mon.a (mon.0) 1163 : audit [INF] from='client.? 172.21.15.67:0/3366967636' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi067-54245-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:39.887426+0000 mon.a (mon.0) 1164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54314-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: cluster 2024-02-10T08:29:39.887613+0000 mon.a (mon.0) 1165 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-10T08:29:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:39.892091+0000 mon.c (mon.2) 295 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:40.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:39 smithi067 bash[24488]: audit 2024-02-10T08:29:39.893060+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[17385]: audit 2024-02-10T08:29:39.946431+0000 mon.c (mon.2) 296 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[17385]: audit 2024-02-10T08:29:39.947492+0000 mon.c (mon.2) 297 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[17385]: cluster 2024-02-10T08:29:39.949148+0000 mgr.y (mgr.14547) 301 : cluster [DBG] pgmap v280: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[17385]: audit 2024-02-10T08:29:40.416359+0000 mon.a (mon.0) 1167 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[17385]: audit 2024-02-10T08:29:40.417427+0000 mon.c (mon.2) 298 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[24488]: audit 2024-02-10T08:29:39.946431+0000 mon.c (mon.2) 296 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:29:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[24488]: audit 2024-02-10T08:29:39.947492+0000 mon.c (mon.2) 297 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:29:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[24488]: cluster 2024-02-10T08:29:39.949148+0000 mgr.y (mgr.14547) 301 : cluster [DBG] pgmap v280: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[24488]: audit 2024-02-10T08:29:40.416359+0000 mon.a (mon.0) 1167 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:41 smithi067 bash[24488]: audit 2024-02-10T08:29:40.417427+0000 mon.c (mon.2) 298 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:41 smithi134 bash[18722]: audit 2024-02-10T08:29:39.946431+0000 mon.c (mon.2) 296 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:29:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:41 smithi134 bash[18722]: audit 2024-02-10T08:29:39.947492+0000 mon.c (mon.2) 297 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:29:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:41 smithi134 bash[18722]: cluster 2024-02-10T08:29:39.949148+0000 mgr.y (mgr.14547) 301 : cluster [DBG] pgmap v280: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-10T08:29:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:41 smithi134 bash[18722]: audit 2024-02-10T08:29:40.416359+0000 mon.a (mon.0) 1167 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:41 smithi134 bash[18722]: audit 2024-02-10T08:29:40.417427+0000 mon.c (mon.2) 298 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:42.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:29:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:29:42.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: cluster 2024-02-10T08:29:41.348251+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: cluster 2024-02-10T08:29:41.348893+0000 mon.a (mon.0) 1169 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: audit 2024-02-10T08:29:41.351092+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? 172.21.15.67:0/3715484529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi067-54237-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: audit 2024-02-10T08:29:41.418647+0000 mon.c (mon.2) 299 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: audit 2024-02-10T08:29:41.896567+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: audit 2024-02-10T08:29:41.897052+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? 172.21.15.67:0/3715484529' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi067-54237-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: cluster 2024-02-10T08:29:41.897300+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-10T08:29:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:42 smithi134 bash[18722]: audit 2024-02-10T08:29:41.907698+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.67:0/1892661173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: cluster 2024-02-10T08:29:41.348251+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-10T08:29:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: cluster 2024-02-10T08:29:41.348893+0000 mon.a (mon.0) 1169 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-02-10T08:29:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: audit 2024-02-10T08:29:41.351092+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? 172.21.15.67:0/3715484529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi067-54237-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: audit 2024-02-10T08:29:41.418647+0000 mon.c (mon.2) 299 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: audit 2024-02-10T08:29:41.896567+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: audit 2024-02-10T08:29:41.897052+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? 172.21.15.67:0/3715484529' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi067-54237-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: cluster 2024-02-10T08:29:41.897300+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[17385]: audit 2024-02-10T08:29:41.907698+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.67:0/1892661173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: cluster 2024-02-10T08:29:41.348251+0000 mon.a (mon.0) 1168 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: cluster 2024-02-10T08:29:41.348893+0000 mon.a (mon.0) 1169 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: audit 2024-02-10T08:29:41.351092+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? 172.21.15.67:0/3715484529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi067-54237-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: audit 2024-02-10T08:29:41.418647+0000 mon.c (mon.2) 299 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: audit 2024-02-10T08:29:41.896567+0000 mon.a (mon.0) 1171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54314-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: audit 2024-02-10T08:29:41.897052+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? 172.21.15.67:0/3715484529' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi067-54237-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: cluster 2024-02-10T08:29:41.897300+0000 mon.a (mon.0) 1173 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-10T08:29:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:42 smithi067 bash[24488]: audit 2024-02-10T08:29:41.907698+0000 mon.c (mon.2) 300 : audit [INF] from='client.? 172.21.15.67:0/1892661173' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.424 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:29:43.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: audit 2024-02-10T08:29:41.907006+0000 mgr.y (mgr.14547) 302 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: audit 2024-02-10T08:29:41.910465+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: cluster 2024-02-10T08:29:41.949907+0000 mgr.y (mgr.14547) 303 : cluster [DBG] pgmap v283: 340 pgs: 40 unknown, 32 creating+peering, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 667 B/s wr, 2 op/s 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: audit 2024-02-10T08:29:42.420194+0000 mon.c (mon.2) 301 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: audit 2024-02-10T08:29:42.901670+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: cluster 2024-02-10T08:29:42.902257+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: audit 2024-02-10T08:29:42.910195+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:43 smithi134 bash[18722]: audit 2024-02-10T08:29:42.913702+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: audit 2024-02-10T08:29:41.907006+0000 mgr.y (mgr.14547) 302 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: audit 2024-02-10T08:29:41.910465+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: cluster 2024-02-10T08:29:41.949907+0000 mgr.y (mgr.14547) 303 : cluster [DBG] pgmap v283: 340 pgs: 40 unknown, 32 creating+peering, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 667 B/s wr, 2 op/s 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: audit 2024-02-10T08:29:42.420194+0000 mon.c (mon.2) 301 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: audit 2024-02-10T08:29:42.901670+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: cluster 2024-02-10T08:29:42.902257+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-10T08:29:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: audit 2024-02-10T08:29:42.910195+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[17385]: audit 2024-02-10T08:29:42.913702+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: audit 2024-02-10T08:29:41.907006+0000 mgr.y (mgr.14547) 302 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: audit 2024-02-10T08:29:41.910465+0000 mon.a (mon.0) 1174 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: cluster 2024-02-10T08:29:41.949907+0000 mgr.y (mgr.14547) 303 : cluster [DBG] pgmap v283: 340 pgs: 40 unknown, 32 creating+peering, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 667 B/s wr, 2 op/s 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: audit 2024-02-10T08:29:42.420194+0000 mon.c (mon.2) 301 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: audit 2024-02-10T08:29:42.901670+0000 mon.a (mon.0) 1175 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi067-54245-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: cluster 2024-02-10T08:29:42.902257+0000 mon.a (mon.0) 1176 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: audit 2024-02-10T08:29:42.910195+0000 mon.c (mon.2) 302 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:43.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:43 smithi067 bash[24488]: audit 2024-02-10T08:29:42.913702+0000 mon.a (mon.0) 1177 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: cluster 2024-02-10T08:29:43.411488+0000 mon.a (mon.0) 1178 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: audit 2024-02-10T08:29:43.421598+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: audit 2024-02-10T08:29:43.906592+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: cluster 2024-02-10T08:29:43.907471+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: audit 2024-02-10T08:29:43.911215+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: audit 2024-02-10T08:29:43.912056+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.67:0/3211734353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: audit 2024-02-10T08:29:43.912795+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:44 smithi134 bash[18722]: audit 2024-02-10T08:29:43.913597+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: cluster 2024-02-10T08:29:43.411488+0000 mon.a (mon.0) 1178 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:44.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: audit 2024-02-10T08:29:43.421598+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: audit 2024-02-10T08:29:43.906592+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: cluster 2024-02-10T08:29:43.907471+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: audit 2024-02-10T08:29:43.911215+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: audit 2024-02-10T08:29:43.912056+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.67:0/3211734353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: audit 2024-02-10T08:29:43.912795+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[17385]: audit 2024-02-10T08:29:43.913597+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: cluster 2024-02-10T08:29:43.411488+0000 mon.a (mon.0) 1178 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: audit 2024-02-10T08:29:43.421598+0000 mon.c (mon.2) 303 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: audit 2024-02-10T08:29:43.906592+0000 mon.a (mon.0) 1179 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: cluster 2024-02-10T08:29:43.907471+0000 mon.a (mon.0) 1180 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: audit 2024-02-10T08:29:43.911215+0000 mon.c (mon.2) 304 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: audit 2024-02-10T08:29:43.912056+0000 mon.c (mon.2) 305 : audit [INF] from='client.? 172.21.15.67:0/3211734353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: audit 2024-02-10T08:29:43.912795+0000 mon.a (mon.0) 1181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:44.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:44 smithi067 bash[24488]: audit 2024-02-10T08:29:43.913597+0000 mon.a (mon.0) 1182 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: cluster 2024-02-10T08:29:43.950733+0000 mgr.y (mgr.14547) 304 : cluster [DBG] pgmap v286: 340 pgs: 72 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: audit 2024-02-10T08:29:44.422841+0000 mon.c (mon.2) 306 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: audit 2024-02-10T08:29:44.915335+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: audit 2024-02-10T08:29:44.915478+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: cluster 2024-02-10T08:29:44.915817+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: audit 2024-02-10T08:29:44.921247+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.67:0/329283727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: audit 2024-02-10T08:29:44.922542+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:45 smithi134 bash[18722]: audit 2024-02-10T08:29:45.424641+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: cluster 2024-02-10T08:29:43.950733+0000 mgr.y (mgr.14547) 304 : cluster [DBG] pgmap v286: 340 pgs: 72 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:29:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: audit 2024-02-10T08:29:44.422841+0000 mon.c (mon.2) 306 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: audit 2024-02-10T08:29:44.915335+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: audit 2024-02-10T08:29:44.915478+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: cluster 2024-02-10T08:29:44.915817+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: audit 2024-02-10T08:29:44.921247+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.67:0/329283727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: audit 2024-02-10T08:29:44.922542+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[17385]: audit 2024-02-10T08:29:45.424641+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: cluster 2024-02-10T08:29:43.950733+0000 mgr.y (mgr.14547) 304 : cluster [DBG] pgmap v286: 340 pgs: 72 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: audit 2024-02-10T08:29:44.422841+0000 mon.c (mon.2) 306 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: audit 2024-02-10T08:29:44.915335+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54314-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: audit 2024-02-10T08:29:44.915478+0000 mon.a (mon.0) 1184 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi067-54237-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: cluster 2024-02-10T08:29:44.915817+0000 mon.a (mon.0) 1185 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: audit 2024-02-10T08:29:44.921247+0000 mon.c (mon.2) 307 : audit [INF] from='client.? 172.21.15.67:0/329283727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: audit 2024-02-10T08:29:44.922542+0000 mon.a (mon.0) 1186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:45.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:45 smithi067 bash[24488]: audit 2024-02-10T08:29:45.424641+0000 mon.c (mon.2) 308 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:47.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:46 smithi134 bash[18722]: audit 2024-02-10T08:29:45.712230+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:46 smithi134 bash[18722]: cluster 2024-02-10T08:29:45.712614+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-10T08:29:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:46 smithi134 bash[18722]: audit 2024-02-10T08:29:46.235223+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:46 smithi134 bash[18722]: audit 2024-02-10T08:29:46.426430+0000 mon.c (mon.2) 309 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[17385]: audit 2024-02-10T08:29:45.712230+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[17385]: cluster 2024-02-10T08:29:45.712614+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-10T08:29:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[17385]: audit 2024-02-10T08:29:46.235223+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[17385]: audit 2024-02-10T08:29:46.426430+0000 mon.c (mon.2) 309 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[24488]: audit 2024-02-10T08:29:45.712230+0000 mon.a (mon.0) 1187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi067-54245-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[24488]: cluster 2024-02-10T08:29:45.712614+0000 mon.a (mon.0) 1188 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-10T08:29:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[24488]: audit 2024-02-10T08:29:46.235223+0000 mon.a (mon.0) 1189 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:29:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:46 smithi067 bash[24488]: audit 2024-02-10T08:29:46.426430+0000 mon.c (mon.2) 309 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: cluster 2024-02-10T08:29:45.951751+0000 mgr.y (mgr.14547) 305 : cluster [DBG] pgmap v289: 308 pgs: 40 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: cluster 2024-02-10T08:29:46.714839+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:46.716732+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:46.717618+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.67:0/485985540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:46.721299+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:46.721615+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:46.739242+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:47.428030+0000 mon.c (mon.2) 311 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:47.719614+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:47.719731+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: audit 2024-02-10T08:29:47.719860+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:48.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:47 smithi134 bash[18722]: cluster 2024-02-10T08:29:47.720199+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-10T08:29:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: cluster 2024-02-10T08:29:45.951751+0000 mgr.y (mgr.14547) 305 : cluster [DBG] pgmap v289: 308 pgs: 40 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:29:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: cluster 2024-02-10T08:29:46.714839+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-10T08:29:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:46.716732+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:46.717618+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.67:0/485985540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:46.721299+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:46.721615+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:46.739242+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:47.428030+0000 mon.c (mon.2) 311 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:47.719614+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:47.719731+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: audit 2024-02-10T08:29:47.719860+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17385]: cluster 2024-02-10T08:29:47.720199+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-10T08:29:48.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:47] "GET /metrics HTTP/1.1" 200 194617 "" "Prometheus/2.33.4" 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: cluster 2024-02-10T08:29:45.951751+0000 mgr.y (mgr.14547) 305 : cluster [DBG] pgmap v289: 308 pgs: 40 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: cluster 2024-02-10T08:29:46.714839+0000 mon.a (mon.0) 1190 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:46.716732+0000 mon.c (mon.2) 310 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:46.717618+0000 mon.b (mon.1) 64 : audit [INF] from='client.? 172.21.15.67:0/485985540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:46.721299+0000 mon.a (mon.0) 1191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:46.721615+0000 mon.a (mon.0) 1192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:46.739242+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:47.428030+0000 mon.c (mon.2) 311 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:47.719614+0000 mon.a (mon.0) 1194 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:47.719731+0000 mon.a (mon.0) 1195 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi067-54237-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: audit 2024-02-10T08:29:47.719860+0000 mon.a (mon.0) 1196 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:47 smithi067 bash[24488]: cluster 2024-02-10T08:29:47.720199+0000 mon.a (mon.0) 1197 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-10T08:29:48.734 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: seed 54485 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2173 ms) 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (3592 ms) 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3355 ms) 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-10T08:29:48.735 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2465 ms) 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (3954 ms) 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (15539 ms total) 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4244 ms) 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4328 ms) 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (8366 ms) 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-10T08:29:48.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4321 ms) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2056 ms) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi067-54485-7 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (18410 ms) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (41725 ms total) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5113 ms) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5121 ms total) 2024-02-10T08:29:48.737 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3007 ms) 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2005 ms) 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2021 ms) 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1798 ms) 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8831 ms total) 2024-02-10T08:29:48.738 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:47.726713+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:47.727062+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:47.741977+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:47.743152+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? 172.21.15.67:0/2814390422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54245-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:48.429788+0000 mon.c (mon.2) 313 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:48.721153+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:48.721344+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: audit 2024-02-10T08:29:48.721448+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.67:0/2814390422' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54245-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:48 smithi134 bash[18722]: cluster 2024-02-10T08:29:48.721667+0000 mon.a (mon.0) 1204 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-10T08:29:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:47.726713+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:47.727062+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:47.741977+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:47.743152+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? 172.21.15.67:0/2814390422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54245-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:48.429788+0000 mon.c (mon.2) 313 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:48.721153+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:48.721344+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: audit 2024-02-10T08:29:48.721448+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.67:0/2814390422' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54245-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[17385]: cluster 2024-02-10T08:29:48.721667+0000 mon.a (mon.0) 1204 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:47.726713+0000 mon.a (mon.0) 1198 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:47.727062+0000 mon.c (mon.2) 312 : audit [INF] from='client.? 172.21.15.67:0/4184871768' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:47.741977+0000 mon.a (mon.0) 1199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:47.743152+0000 mon.a (mon.0) 1200 : audit [INF] from='client.? 172.21.15.67:0/2814390422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54245-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:48.429788+0000 mon.c (mon.2) 313 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:48.721153+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? 172.21.15.67:0/3258227729' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi067-54485-15"}]': finished 2024-02-10T08:29:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:48.721344+0000 mon.a (mon.0) 1202 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54314-24"}]': finished 2024-02-10T08:29:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: audit 2024-02-10T08:29:48.721448+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? 172.21.15.67:0/2814390422' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi067-54245-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:48 smithi067 bash[24488]: cluster 2024-02-10T08:29:48.721667+0000 mon.a (mon.0) 1204 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-10T08:29:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: cluster 2024-02-10T08:29:47.952617+0000 mgr.y (mgr.14547) 306 : cluster [DBG] pgmap v292: 324 pgs: 64 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: cluster 2024-02-10T08:29:48.731616+0000 mon.a (mon.0) 1205 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.773683+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.774742+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.775575+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.775797+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.776341+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.776745+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.776989+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.777446+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.777510+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.778074+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.778231+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:48.778712+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:49.431504+0000 mon.c (mon.2) 320 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:50.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:49.763225+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:50.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: audit 2024-02-10T08:29:49.763348+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:50.029 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:49 smithi134 bash[18722]: cluster 2024-02-10T08:29:49.763469+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: cluster 2024-02-10T08:29:47.952617+0000 mgr.y (mgr.14547) 306 : cluster [DBG] pgmap v292: 324 pgs: 64 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: cluster 2024-02-10T08:29:48.731616+0000 mon.a (mon.0) 1205 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.773683+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.774742+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.775575+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.775797+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.776341+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.776745+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.776989+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.777446+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.777510+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.778074+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.778231+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:48.778712+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: cluster 2024-02-10T08:29:47.952617+0000 mgr.y (mgr.14547) 306 : cluster [DBG] pgmap v292: 324 pgs: 64 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: cluster 2024-02-10T08:29:48.731616+0000 mon.a (mon.0) 1205 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.773683+0000 mon.c (mon.2) 314 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.774742+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.775575+0000 mon.c (mon.2) 315 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.775797+0000 mon.c (mon.2) 316 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.776341+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.776745+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.776989+0000 mon.c (mon.2) 317 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.777446+0000 mon.a (mon.0) 1209 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.777510+0000 mon.c (mon.2) 318 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.778074+0000 mon.a (mon.0) 1210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.778231+0000 mon.c (mon.2) 319 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:48.778712+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:49.431504+0000 mon.c (mon.2) 320 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:49.763225+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:50.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: audit 2024-02-10T08:29:49.763348+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:50.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[17385]: cluster 2024-02-10T08:29:49.763469+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-10T08:29:50.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:49.431504+0000 mon.c (mon.2) 320 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:50.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:49.763225+0000 mon.a (mon.0) 1212 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:50.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: audit 2024-02-10T08:29:49.763348+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi067-54237-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:50.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:49 smithi067 bash[24488]: cluster 2024-02-10T08:29:49.763469+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-10T08:29:51.916 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:49.772576+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:49.772876+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:49.773104+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.67:0/1718257153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:49.791720+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:49.793014+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:49.793555+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: cluster 2024-02-10T08:29:49.953457+0000 mgr.y (mgr.14547) 307 : cluster [DBG] pgmap v295: 292 pgs: 32 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:51.917 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[18722]: audit 2024-02-10T08:29:50.433120+0000 mon.c (mon.2) 324 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:49.772576+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:49.772876+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:49.773104+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.67:0/1718257153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:49.791720+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:49.793014+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:49.793555+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: cluster 2024-02-10T08:29:49.953457+0000 mgr.y (mgr.14547) 307 : cluster [DBG] pgmap v295: 292 pgs: 32 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[24488]: audit 2024-02-10T08:29:50.433120+0000 mon.c (mon.2) 324 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:49.772576+0000 mon.c (mon.2) 321 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:49.772876+0000 mon.c (mon.2) 322 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:49.773104+0000 mon.c (mon.2) 323 : audit [INF] from='client.? 172.21.15.67:0/1718257153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:49.791720+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:49.793014+0000 mon.a (mon.0) 1216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:49.793555+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: cluster 2024-02-10T08:29:49.953457+0000 mgr.y (mgr.14547) 307 : cluster [DBG] pgmap v295: 292 pgs: 32 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:29:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:51 smithi067 bash[17385]: audit 2024-02-10T08:29:50.433120+0000 mon.c (mon.2) 324 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:52.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:29:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:51.434813+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:51.649705+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: cluster 2024-02-10T08:29:51.649963+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:51.654698+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? 172.21.15.67:0/126738776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi067-54245-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:51.765826+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:51.766055+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:51.766309+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.67:0/126738776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi067-54245-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: cluster 2024-02-10T08:29:51.766673+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-10T08:29:53.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:52 smithi134 bash[18722]: audit 2024-02-10T08:29:52.436545+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:51.434813+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:51.649705+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: cluster 2024-02-10T08:29:51.649963+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-10T08:29:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:51.654698+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? 172.21.15.67:0/126738776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi067-54245-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:51.765826+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:51.766055+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:51.766309+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.67:0/126738776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi067-54245-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: cluster 2024-02-10T08:29:51.766673+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[24488]: audit 2024-02-10T08:29:52.436545+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:51.434813+0000 mon.c (mon.2) 325 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:51.649705+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi067-54314-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: cluster 2024-02-10T08:29:51.649963+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:51.654698+0000 mon.a (mon.0) 1220 : audit [INF] from='client.? 172.21.15.67:0/126738776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi067-54245-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:51.765826+0000 mon.a (mon.0) 1221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:51.766055+0000 mon.a (mon.0) 1222 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi067-54237-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:51.766309+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? 172.21.15.67:0/126738776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi067-54245-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: cluster 2024-02-10T08:29:51.766673+0000 mon.a (mon.0) 1224 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-10T08:29:53.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:52 smithi067 bash[17385]: audit 2024-02-10T08:29:52.436545+0000 mon.c (mon.2) 326 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:29:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[18722]: audit 2024-02-10T08:29:51.915182+0000 mgr.y (mgr.14547) 308 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[18722]: cluster 2024-02-10T08:29:51.954272+0000 mgr.y (mgr.14547) 309 : cluster [DBG] pgmap v298: 308 pgs: 28 creating+peering, 20 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:29:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[18722]: cluster 2024-02-10T08:29:52.774101+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-10T08:29:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[18722]: audit 2024-02-10T08:29:52.782440+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.67:0/1268275371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[18722]: audit 2024-02-10T08:29:52.788821+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:54.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:53 smithi134 bash[18722]: audit 2024-02-10T08:29:53.438270+0000 mon.c (mon.2) 328 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[17385]: audit 2024-02-10T08:29:51.915182+0000 mgr.y (mgr.14547) 308 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[17385]: cluster 2024-02-10T08:29:51.954272+0000 mgr.y (mgr.14547) 309 : cluster [DBG] pgmap v298: 308 pgs: 28 creating+peering, 20 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:29:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[17385]: cluster 2024-02-10T08:29:52.774101+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-10T08:29:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[17385]: audit 2024-02-10T08:29:52.782440+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.67:0/1268275371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[17385]: audit 2024-02-10T08:29:52.788821+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[17385]: audit 2024-02-10T08:29:53.438270+0000 mon.c (mon.2) 328 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[24488]: audit 2024-02-10T08:29:51.915182+0000 mgr.y (mgr.14547) 308 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[24488]: cluster 2024-02-10T08:29:51.954272+0000 mgr.y (mgr.14547) 309 : cluster [DBG] pgmap v298: 308 pgs: 28 creating+peering, 20 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[24488]: cluster 2024-02-10T08:29:52.774101+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[24488]: audit 2024-02-10T08:29:52.782440+0000 mon.c (mon.2) 327 : audit [INF] from='client.? 172.21.15.67:0/1268275371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[24488]: audit 2024-02-10T08:29:52.788821+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:54.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:53 smithi067 bash[24488]: audit 2024-02-10T08:29:53.438270+0000 mon.c (mon.2) 328 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:55.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: audit 2024-02-10T08:29:53.780068+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:55.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: cluster 2024-02-10T08:29:53.781465+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: audit 2024-02-10T08:29:53.786404+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: audit 2024-02-10T08:29:53.786752+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.67:0/3135832894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: audit 2024-02-10T08:29:53.787189+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: audit 2024-02-10T08:29:53.790489+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: cluster 2024-02-10T08:29:53.955150+0000 mgr.y (mgr.14547) 310 : cluster [DBG] pgmap v301: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[17385]: audit 2024-02-10T08:29:54.440085+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: audit 2024-02-10T08:29:53.780068+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: cluster 2024-02-10T08:29:53.781465+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: audit 2024-02-10T08:29:53.786404+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: audit 2024-02-10T08:29:53.786752+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.67:0/3135832894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:55.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: audit 2024-02-10T08:29:53.787189+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:55.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: audit 2024-02-10T08:29:53.790489+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:55.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: cluster 2024-02-10T08:29:53.955150+0000 mgr.y (mgr.14547) 310 : cluster [DBG] pgmap v301: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:29:55.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:54 smithi067 bash[24488]: audit 2024-02-10T08:29:54.440085+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: audit 2024-02-10T08:29:53.780068+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi067-54314-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: cluster 2024-02-10T08:29:53.781465+0000 mon.a (mon.0) 1228 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: audit 2024-02-10T08:29:53.786404+0000 mon.c (mon.2) 329 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: audit 2024-02-10T08:29:53.786752+0000 mon.c (mon.2) 330 : audit [INF] from='client.? 172.21.15.67:0/3135832894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: audit 2024-02-10T08:29:53.787189+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: audit 2024-02-10T08:29:53.790489+0000 mon.a (mon.0) 1230 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: cluster 2024-02-10T08:29:53.955150+0000 mgr.y (mgr.14547) 310 : cluster [DBG] pgmap v301: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:29:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:54 smithi134 bash[18722]: audit 2024-02-10T08:29:54.440085+0000 mon.c (mon.2) 331 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:55.798 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [----------] Global test environment set-up. 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.TooBig (11174 ms) 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7277 ms) 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (6292 ms) 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2533 ms) 2024-02-10T08:29:55.799 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3343 ms) 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3257 ms) 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3151 ms) 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (4010 ms) 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2397 ms) 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3193 ms) 2024-02-10T08:29:55.800 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3309 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3013 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3449 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2528 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.Flush (3047 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2966 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3012 ms) 2024-02-10T08:29:55.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3019 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3034 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3044 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3062 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3011 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3025 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (2799 ms) 2024-02-10T08:29:55.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-10T08:29:55.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3011 ms) 2024-02-10T08:29:55.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [----------] 25 tests from LibRadosAio (91957 ms total) 2024-02-10T08:29:55.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: 2024-02-10T08:29:55.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-10T08:29:55.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-10T08:29:55.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7069 ms) 2024-02-10T08:29:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:54.782239+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:54.782438+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: cluster 2024-02-10T08:29:54.782946+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-10T08:29:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:54.783880+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:54.804226+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:55.441926+0000 mon.c (mon.2) 333 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:55.785898+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: cluster 2024-02-10T08:29:55.786256+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[17385]: audit 2024-02-10T08:29:55.797548+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.67:0/118450333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi067-54314-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:54.782239+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:54.782438+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: cluster 2024-02-10T08:29:54.782946+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:54.783880+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:54.804226+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:55.441926+0000 mon.c (mon.2) 333 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:55.785898+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: cluster 2024-02-10T08:29:55.786256+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-10T08:29:56.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:55 smithi067 bash[24488]: audit 2024-02-10T08:29:55.797548+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.67:0/118450333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi067-54314-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:54.782239+0000 mon.a (mon.0) 1231 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:54.782438+0000 mon.a (mon.0) 1232 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi067-54245-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: cluster 2024-02-10T08:29:54.782946+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:54.783880+0000 mon.c (mon.2) 332 : audit [INF] from='client.? 172.21.15.67:0/1409753573' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:54.804226+0000 mon.a (mon.0) 1234 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]: dispatch 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:55.441926+0000 mon.c (mon.2) 333 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:55.785898+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi067-54237-26"}]': finished 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: cluster 2024-02-10T08:29:55.786256+0000 mon.a (mon.0) 1236 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-10T08:29:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:55 smithi134 bash[18722]: audit 2024-02-10T08:29:55.797548+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? 172.21.15.67:0/118450333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi067-54314-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:55.831003+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: cluster 2024-02-10T08:29:55.955965+0000 mgr.y (mgr.14547) 311 : cluster [DBG] pgmap v304: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.288436+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.289621+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.292664+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.293864+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.294590+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.443585+0000 mon.c (mon.2) 337 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.792536+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? 172.21.15.67:0/118450333' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi067-54314-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.792905+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: cluster 2024-02-10T08:29:56.793018+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.801895+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.802832+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? 172.21.15.67:0/1122167033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi067-54245-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[17385]: audit 2024-02-10T08:29:56.803909+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:55.831003+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: cluster 2024-02-10T08:29:55.955965+0000 mgr.y (mgr.14547) 311 : cluster [DBG] pgmap v304: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.288436+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.289621+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.292664+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.293864+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.294590+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.443585+0000 mon.c (mon.2) 337 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.792536+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? 172.21.15.67:0/118450333' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi067-54314-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.792905+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: cluster 2024-02-10T08:29:56.793018+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-10T08:29:57.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.801895+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.802832+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? 172.21.15.67:0/1122167033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi067-54245-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:57.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:56 smithi067 bash[24488]: audit 2024-02-10T08:29:56.803909+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:55.831003+0000 mon.c (mon.2) 334 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: cluster 2024-02-10T08:29:55.955965+0000 mgr.y (mgr.14547) 311 : cluster [DBG] pgmap v304: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:29:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.288436+0000 mon.a (mon.0) 1238 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.289621+0000 mon.c (mon.2) 335 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.292664+0000 mon.a (mon.0) 1239 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.293864+0000 mon.c (mon.2) 336 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.294590+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.443585+0000 mon.c (mon.2) 337 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.792536+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? 172.21.15.67:0/118450333' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi067-54314-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.792905+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi067-54237-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: cluster 2024-02-10T08:29:56.793018+0000 mon.a (mon.0) 1243 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.801895+0000 mon.c (mon.2) 338 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.802832+0000 mon.a (mon.0) 1244 : audit [INF] from='client.? 172.21.15.67:0/1122167033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi067-54245-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:29:57.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:56 smithi134 bash[18722]: audit 2024-02-10T08:29:56.803909+0000 mon.a (mon.0) 1245 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:29:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:29:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:29:57] "GET /metrics HTTP/1.1" 200 194617 "" "Prometheus/2.33.4" 2024-02-10T08:29:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:58 smithi134 bash[18722]: audit 2024-02-10T08:29:57.445418+0000 mon.c (mon.2) 339 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:58 smithi067 bash[17385]: audit 2024-02-10T08:29:57.445418+0000 mon.c (mon.2) 339 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:58.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:58 smithi067 bash[24488]: audit 2024-02-10T08:29:57.445418+0000 mon.c (mon.2) 339 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:57.796031+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.67:0/1122167033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi067-54245-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: cluster 2024-02-10T08:29:57.956823+0000 mgr.y (mgr.14547) 312 : cluster [DBG] pgmap v307: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: cluster 2024-02-10T08:29:58.266246+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.293997+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.297879+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.298433+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.447173+0000 mon.c (mon.2) 340 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.796599+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.796832+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: cluster 2024-02-10T08:29:58.796931+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-10T08:29:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[17385]: audit 2024-02-10T08:29:58.825526+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi067-54314-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:57.796031+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.67:0/1122167033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi067-54245-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: cluster 2024-02-10T08:29:57.956823+0000 mgr.y (mgr.14547) 312 : cluster [DBG] pgmap v307: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: cluster 2024-02-10T08:29:58.266246+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.293997+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.297879+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.298433+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.447173+0000 mon.c (mon.2) 340 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.796599+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.796832+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: cluster 2024-02-10T08:29:58.796931+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-10T08:29:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:29:59 smithi067 bash[24488]: audit 2024-02-10T08:29:58.825526+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi067-54314-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:57.796031+0000 mon.a (mon.0) 1246 : audit [INF] from='client.? 172.21.15.67:0/1122167033' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi067-54245-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: cluster 2024-02-10T08:29:57.956823+0000 mgr.y (mgr.14547) 312 : cluster [DBG] pgmap v307: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: cluster 2024-02-10T08:29:58.266246+0000 mon.a (mon.0) 1247 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.293997+0000 mon.a (mon.0) 1248 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.297879+0000 mon.a (mon.0) 1249 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.298433+0000 mon.a (mon.0) 1250 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.447173+0000 mon.c (mon.2) 340 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.796599+0000 mon.a (mon.0) 1251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi067-54237-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.796832+0000 mon.a (mon.0) 1252 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:29:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: cluster 2024-02-10T08:29:58.796931+0000 mon.a (mon.0) 1253 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-10T08:29:59.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:29:59 smithi134 bash[18722]: audit 2024-02-10T08:29:58.825526+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi067-54314-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: audit 2024-02-10T08:29:59.448926+0000 mon.c (mon.2) 341 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.270107+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.270893+0000 mon.a (mon.0) 1256 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.271021+0000 mon.a (mon.0) 1257 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.271095+0000 mon.a (mon.0) 1258 : cluster [WRN] pool 'test-rados-api-smithi067-54339-13' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.271153+0000 mon.a (mon.0) 1259 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.271210+0000 mon.a (mon.0) 1260 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.271250+0000 mon.a (mon.0) 1261 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[17385]: audit 2024-02-10T08:30:00.275610+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? 172.21.15.67:0/3219624812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi067-54245-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: audit 2024-02-10T08:29:59.448926+0000 mon.c (mon.2) 341 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.270107+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.270893+0000 mon.a (mon.0) 1256 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.271021+0000 mon.a (mon.0) 1257 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.271095+0000 mon.a (mon.0) 1258 : cluster [WRN] pool 'test-rados-api-smithi067-54339-13' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.271153+0000 mon.a (mon.0) 1259 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.271210+0000 mon.a (mon.0) 1260 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.271250+0000 mon.a (mon.0) 1261 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T08:30:00.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:00 smithi067 bash[24488]: audit 2024-02-10T08:30:00.275610+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? 172.21.15.67:0/3219624812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi067-54245-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: audit 2024-02-10T08:29:59.448926+0000 mon.c (mon.2) 341 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.270107+0000 mon.a (mon.0) 1255 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.270893+0000 mon.a (mon.0) 1256 : cluster [WRN] Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.271021+0000 mon.a (mon.0) 1257 : cluster [WRN] [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.271095+0000 mon.a (mon.0) 1258 : cluster [WRN] pool 'test-rados-api-smithi067-54339-13' with cache_mode writeback needs hit_set_type to be set but it is not 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.271153+0000 mon.a (mon.0) 1259 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.271210+0000 mon.a (mon.0) 1260 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.271250+0000 mon.a (mon.0) 1261 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T08:30:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:00 smithi134 bash[18722]: audit 2024-02-10T08:30:00.275610+0000 mon.a (mon.0) 1262 : audit [INF] from='client.? 172.21.15.67:0/3219624812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi067-54245-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: cluster 2024-02-10T08:29:59.957613+0000 mgr.y (mgr.14547) 313 : cluster [DBG] pgmap v310: 308 pgs: 40 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.450707+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.494319+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.494930+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.721227+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi067-54314-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.721632+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.67:0/3219624812' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi067-54245-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.722339+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: cluster 2024-02-10T08:30:00.722472+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.723913+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.725698+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.732638+0000 mon.a (mon.0) 1268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:01.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[17385]: audit 2024-02-10T08:30:00.733247+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: cluster 2024-02-10T08:29:59.957613+0000 mgr.y (mgr.14547) 313 : cluster [DBG] pgmap v310: 308 pgs: 40 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.450707+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.494319+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.494930+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.721227+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi067-54314-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.721632+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.67:0/3219624812' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi067-54245-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.722339+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: cluster 2024-02-10T08:30:00.722472+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.723913+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.725698+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.732638+0000 mon.a (mon.0) 1268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:01.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:01 smithi067 bash[24488]: audit 2024-02-10T08:30:00.733247+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:30:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: cluster 2024-02-10T08:29:59.957613+0000 mgr.y (mgr.14547) 313 : cluster [DBG] pgmap v310: 308 pgs: 40 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-02-10T08:30:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.450707+0000 mon.c (mon.2) 342 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.494319+0000 mon.c (mon.2) 343 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.494930+0000 mon.a (mon.0) 1263 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:01.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.721227+0000 mon.a (mon.0) 1264 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi067-54314-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.721632+0000 mon.a (mon.0) 1265 : audit [INF] from='client.? 172.21.15.67:0/3219624812' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi067-54245-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.722339+0000 mon.a (mon.0) 1266 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: cluster 2024-02-10T08:30:00.722472+0000 mon.a (mon.0) 1267 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.723913+0000 mon.c (mon.2) 344 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.725698+0000 mon.c (mon.2) 345 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.732638+0000 mon.a (mon.0) 1268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:01.781 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[18722]: audit 2024-02-10T08:30:00.733247+0000 mon.a (mon.0) 1269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]: dispatch 2024-02-10T08:30:02.282 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:30:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: audit 2024-02-10T08:30:01.452468+0000 mon.c (mon.2) 346 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: cluster 2024-02-10T08:30:01.713636+0000 mon.a (mon.0) 1270 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: audit 2024-02-10T08:30:01.722283+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: audit 2024-02-10T08:30:01.722489+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]': finished 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: cluster 2024-02-10T08:30:01.723036+0000 mon.a (mon.0) 1273 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: audit 2024-02-10T08:30:01.723576+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:02.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[17385]: audit 2024-02-10T08:30:01.724527+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: audit 2024-02-10T08:30:01.452468+0000 mon.c (mon.2) 346 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: cluster 2024-02-10T08:30:01.713636+0000 mon.a (mon.0) 1270 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: audit 2024-02-10T08:30:01.722283+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: audit 2024-02-10T08:30:01.722489+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]': finished 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: cluster 2024-02-10T08:30:01.723036+0000 mon.a (mon.0) 1273 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: audit 2024-02-10T08:30:01.723576+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:02.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:02 smithi067 bash[24488]: audit 2024-02-10T08:30:01.724527+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:02.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: audit 2024-02-10T08:30:01.452468+0000 mon.c (mon.2) 346 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: cluster 2024-02-10T08:30:01.713636+0000 mon.a (mon.0) 1270 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: audit 2024-02-10T08:30:01.722283+0000 mon.a (mon.0) 1271 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:30:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: audit 2024-02-10T08:30:01.722489+0000 mon.a (mon.0) 1272 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-13"}]': finished 2024-02-10T08:30:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: cluster 2024-02-10T08:30:01.723036+0000 mon.a (mon.0) 1273 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-10T08:30:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: audit 2024-02-10T08:30:01.723576+0000 mon.c (mon.2) 347 : audit [INF] from='client.? 172.21.15.67:0/3724775632' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:02 smithi134 bash[18722]: audit 2024-02-10T08:30:01.724527+0000 mon.a (mon.0) 1274 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]: dispatch 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:01.923343+0000 mgr.y (mgr.14547) 314 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: cluster 2024-02-10T08:30:01.958465+0000 mgr.y (mgr.14547) 315 : cluster [DBG] pgmap v313: 276 pgs: 8 creating+peering, 268 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:02.454237+0000 mon.c (mon.2) 348 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: cluster 2024-02-10T08:30:02.716129+0000 mon.a (mon.0) 1275 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:02.733608+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: cluster 2024-02-10T08:30:02.733873+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-10T08:30:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:02.742020+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.67:0/3136127786' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:02.778752+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.201528+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.202549+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.204995+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.206299+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.206860+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.207646+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:03.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:03 smithi134 bash[18722]: audit 2024-02-10T08:30:03.208109+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:01.923343+0000 mgr.y (mgr.14547) 314 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: cluster 2024-02-10T08:30:01.958465+0000 mgr.y (mgr.14547) 315 : cluster [DBG] pgmap v313: 276 pgs: 8 creating+peering, 268 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:02.454237+0000 mon.c (mon.2) 348 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: cluster 2024-02-10T08:30:02.716129+0000 mon.a (mon.0) 1275 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:02.733608+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: cluster 2024-02-10T08:30:02.733873+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:02.742020+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.67:0/3136127786' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:02.778752+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.201528+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.202549+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.204995+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.206299+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.206860+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.207646+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[17385]: audit 2024-02-10T08:30:03.208109+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:01.923343+0000 mgr.y (mgr.14547) 314 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: cluster 2024-02-10T08:30:01.958465+0000 mgr.y (mgr.14547) 315 : cluster [DBG] pgmap v313: 276 pgs: 8 creating+peering, 268 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:02.454237+0000 mon.c (mon.2) 348 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: cluster 2024-02-10T08:30:02.716129+0000 mon.a (mon.0) 1275 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:02.733608+0000 mon.a (mon.0) 1276 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi067-54237-27"}]': finished 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: cluster 2024-02-10T08:30:02.733873+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:02.742020+0000 mon.c (mon.2) 349 : audit [INF] from='client.? 172.21.15.67:0/3136127786' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:02.778752+0000 mon.c (mon.2) 350 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.201528+0000 mon.a (mon.0) 1278 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:03.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.202549+0000 mon.a (mon.0) 1279 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:03.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.204995+0000 mon.a (mon.0) 1280 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.206299+0000 mon.c (mon.2) 351 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.206860+0000 mon.a (mon.0) 1281 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:03.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.207646+0000 mon.c (mon.2) 352 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:03.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:03 smithi067 bash[24488]: audit 2024-02-10T08:30:03.208109+0000 mon.a (mon.0) 1282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.455992+0000 mon.c (mon.2) 353 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.739649+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.739871+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.740045+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: cluster 2024-02-10T08:30:03.740140+0000 mon.a (mon.0) 1286 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.746355+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.762876+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.763980+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.764321+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[17385]: audit 2024-02-10T08:30:03.765081+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.455992+0000 mon.c (mon.2) 353 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.739649+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.739871+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.740045+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: cluster 2024-02-10T08:30:03.740140+0000 mon.a (mon.0) 1286 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.746355+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.762876+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.763980+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.764321+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:04.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:04 smithi067 bash[24488]: audit 2024-02-10T08:30:03.765081+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:04.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.455992+0000 mon.c (mon.2) 353 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.739649+0000 mon.a (mon.0) 1283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi067-54245-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.739871+0000 mon.a (mon.0) 1284 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.740045+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi067-54237-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: cluster 2024-02-10T08:30:03.740140+0000 mon.a (mon.0) 1286 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.746355+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]: dispatch 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.762876+0000 mon.c (mon.2) 354 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.763980+0000 mon.c (mon.2) 355 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.764321+0000 mon.a (mon.0) 1288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:04 smithi134 bash[18722]: audit 2024-02-10T08:30:03.765081+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:05.204 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompl(69356 ms total) 2024-02-10T08:30:05.204 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:30:05.204 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-10T08:30:05.204 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-10T08:30:05.204 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (137 ms) 2024-02-10T08:30:05.204 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (137 ms total) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (9 ms) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (6 ms) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (15 ms total) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (21 ms) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (4 ms) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (25 ms total) 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:30:05.205 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (48 ms) 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (4 ms) 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (52 ms total) 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1017 ms) 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1018 ms total) 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (108381 ms total) 2024-02-10T08:30:05.206 INFO:tasks.workunit.client.0.smithi067.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: cluster 2024-02-10T08:30:03.959445+0000 mgr.y (mgr.14547) 316 : cluster [DBG] pgmap v316: 300 pgs: 64 unknown, 236 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: audit 2024-02-10T08:30:04.457660+0000 mon.c (mon.2) 356 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: audit 2024-02-10T08:30:04.745085+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: audit 2024-02-10T08:30:04.745330+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: audit 2024-02-10T08:30:04.752596+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: cluster 2024-02-10T08:30:05.196858+0000 mon.a (mon.0) 1292 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-10T08:30:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:05 smithi134 bash[18722]: audit 2024-02-10T08:30:05.198477+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: cluster 2024-02-10T08:30:03.959445+0000 mgr.y (mgr.14547) 316 : cluster [DBG] pgmap v316: 300 pgs: 64 unknown, 236 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T08:30:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: audit 2024-02-10T08:30:04.457660+0000 mon.c (mon.2) 356 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: audit 2024-02-10T08:30:04.745085+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: audit 2024-02-10T08:30:04.745330+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: audit 2024-02-10T08:30:04.752596+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: cluster 2024-02-10T08:30:05.196858+0000 mon.a (mon.0) 1292 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[17385]: audit 2024-02-10T08:30:05.198477+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: cluster 2024-02-10T08:30:03.959445+0000 mgr.y (mgr.14547) 316 : cluster [DBG] pgmap v316: 300 pgs: 64 unknown, 236 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: audit 2024-02-10T08:30:04.457660+0000 mon.c (mon.2) 356 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: audit 2024-02-10T08:30:04.745085+0000 mon.a (mon.0) 1290 : audit [INF] from='client.? 172.21.15.67:0/3153671536' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi067-54314-36"}]': finished 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: audit 2024-02-10T08:30:04.745330+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: audit 2024-02-10T08:30:04.752596+0000 mon.c (mon.2) 357 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: cluster 2024-02-10T08:30:05.196858+0000 mon.a (mon.0) 1292 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-10T08:30:06.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:05 smithi067 bash[24488]: audit 2024-02-10T08:30:05.198477+0000 mon.a (mon.0) 1293 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:06.735 INFO:tasks.workunit.client.0.smithi067.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4035 ms) 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (4454 ms) 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3490 ms) 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11979 ms total) 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (109876 ms total) 2024-02-10T08:30:06.736 INFO:tasks.workunit.client.0.smithi067.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.459289+0000 mon.c (mon.2) 358 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.721417+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.721552+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: cluster 2024-02-10T08:30:05.721723+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.725417+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.725621+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.67:0/3673831596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.735300+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.736707+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.742508+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:05.742898+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:06 smithi134 bash[18722]: audit 2024-02-10T08:30:06.460923+0000 mon.c (mon.2) 362 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:07.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.459289+0000 mon.c (mon.2) 358 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:07.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.721417+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:07.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.721552+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:30:07.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: cluster 2024-02-10T08:30:05.721723+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.725417+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.725621+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.67:0/3673831596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.735300+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.736707+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.742508+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:05.742898+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[17385]: audit 2024-02-10T08:30:06.460923+0000 mon.c (mon.2) 362 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.459289+0000 mon.c (mon.2) 358 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.721417+0000 mon.a (mon.0) 1294 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi067-54237-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.721552+0000 mon.a (mon.0) 1295 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: cluster 2024-02-10T08:30:05.721723+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.725417+0000 mon.c (mon.2) 359 : audit [INF] from='client.? 172.21.15.67:0/506773530' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.725621+0000 mon.c (mon.2) 360 : audit [INF] from='client.? 172.21.15.67:0/3673831596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:07.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.735300+0000 mon.a (mon.0) 1297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]: dispatch 2024-02-10T08:30:07.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.736707+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:07.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.742508+0000 mon.c (mon.2) 361 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:07.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:05.742898+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:07.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:06 smithi067 bash[24488]: audit 2024-02-10T08:30:06.460923+0000 mon.c (mon.2) 362 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: cluster 2024-02-10T08:30:05.960328+0000 mgr.y (mgr.14547) 317 : cluster [DBG] pgmap v319: 300 pgs: 72 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: audit 2024-02-10T08:30:06.725631+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: audit 2024-02-10T08:30:06.725829+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: audit 2024-02-10T08:30:06.726053+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: cluster 2024-02-10T08:30:06.726196+0000 mon.a (mon.0) 1303 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: audit 2024-02-10T08:30:06.732029+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: audit 2024-02-10T08:30:06.739507+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:07 smithi134 bash[18722]: audit 2024-02-10T08:30:07.462658+0000 mon.c (mon.2) 364 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:07] "GET /metrics HTTP/1.1" 200 194984 "" "Prometheus/2.33.4" 2024-02-10T08:30:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: cluster 2024-02-10T08:30:05.960328+0000 mgr.y (mgr.14547) 317 : cluster [DBG] pgmap v319: 300 pgs: 72 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: audit 2024-02-10T08:30:06.725631+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: audit 2024-02-10T08:30:06.725829+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: audit 2024-02-10T08:30:06.726053+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: cluster 2024-02-10T08:30:06.726196+0000 mon.a (mon.0) 1303 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: audit 2024-02-10T08:30:06.732029+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: audit 2024-02-10T08:30:06.739507+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[17385]: audit 2024-02-10T08:30:07.462658+0000 mon.c (mon.2) 364 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: cluster 2024-02-10T08:30:05.960328+0000 mgr.y (mgr.14547) 317 : cluster [DBG] pgmap v319: 300 pgs: 72 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: audit 2024-02-10T08:30:06.725631+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi067-54485-20"}]': finished 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: audit 2024-02-10T08:30:06.725829+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi067-54245-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: audit 2024-02-10T08:30:06.726053+0000 mon.a (mon.0) 1302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: cluster 2024-02-10T08:30:06.726196+0000 mon.a (mon.0) 1303 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-10T08:30:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: audit 2024-02-10T08:30:06.732029+0000 mon.c (mon.2) 363 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:08.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: audit 2024-02-10T08:30:06.739507+0000 mon.a (mon.0) 1304 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:08.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:07 smithi067 bash[24488]: audit 2024-02-10T08:30:07.462658+0000 mon.c (mon.2) 364 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: audit 2024-02-10T08:30:07.741034+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]': finished 2024-02-10T08:30:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: cluster 2024-02-10T08:30:07.741127+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-10T08:30:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: audit 2024-02-10T08:30:07.751257+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]: dispatch 2024-02-10T08:30:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: audit 2024-02-10T08:30:07.752871+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]: dispatch 2024-02-10T08:30:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: audit 2024-02-10T08:30:07.753141+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: audit 2024-02-10T08:30:07.753608+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:08 smithi134 bash[18722]: audit 2024-02-10T08:30:08.464588+0000 mon.c (mon.2) 367 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: audit 2024-02-10T08:30:07.741034+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]': finished 2024-02-10T08:30:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: cluster 2024-02-10T08:30:07.741127+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-10T08:30:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: audit 2024-02-10T08:30:07.751257+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]: dispatch 2024-02-10T08:30:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: audit 2024-02-10T08:30:07.752871+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]: dispatch 2024-02-10T08:30:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: audit 2024-02-10T08:30:07.753141+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: audit 2024-02-10T08:30:07.753608+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[24488]: audit 2024-02-10T08:30:08.464588+0000 mon.c (mon.2) 367 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: audit 2024-02-10T08:30:07.741034+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-15"}]': finished 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: cluster 2024-02-10T08:30:07.741127+0000 mon.a (mon.0) 1306 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: audit 2024-02-10T08:30:07.751257+0000 mon.c (mon.2) 365 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: audit 2024-02-10T08:30:07.752871+0000 mon.a (mon.0) 1307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: audit 2024-02-10T08:30:07.753141+0000 mon.c (mon.2) 366 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: audit 2024-02-10T08:30:07.753608+0000 mon.a (mon.0) 1308 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:08 smithi067 bash[17385]: audit 2024-02-10T08:30:08.464588+0000 mon.c (mon.2) 367 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: cluster 2024-02-10T08:30:07.961131+0000 mgr.y (mgr.14547) 318 : cluster [DBG] pgmap v322: 260 pgs: 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: cluster 2024-02-10T08:30:08.740291+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:08.759770+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]': finished 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:08.759924+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: cluster 2024-02-10T08:30:08.760062+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:08.764445+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:08.765509+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.67:0/2585205636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:08.768760+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:08.770016+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:09.466552+0000 mon.c (mon.2) 370 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:09.467924+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:30:10.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:09 smithi134 bash[18722]: audit 2024-02-10T08:30:09.468773+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:30:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: cluster 2024-02-10T08:30:07.961131+0000 mgr.y (mgr.14547) 318 : cluster [DBG] pgmap v322: 260 pgs: 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: cluster 2024-02-10T08:30:08.740291+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:08.759770+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]': finished 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:08.759924+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: cluster 2024-02-10T08:30:08.760062+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:08.764445+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:08.765509+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.67:0/2585205636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:08.768760+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:08.770016+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:09.466552+0000 mon.c (mon.2) 370 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:09.467924+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:30:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[17385]: audit 2024-02-10T08:30:09.468773+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: cluster 2024-02-10T08:30:07.961131+0000 mgr.y (mgr.14547) 318 : cluster [DBG] pgmap v322: 260 pgs: 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: cluster 2024-02-10T08:30:08.740291+0000 mon.a (mon.0) 1309 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:08.759770+0000 mon.a (mon.0) 1310 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-15", "mode": "writeback"}]': finished 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:08.759924+0000 mon.a (mon.0) 1311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: cluster 2024-02-10T08:30:08.760062+0000 mon.a (mon.0) 1312 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:08.764445+0000 mon.c (mon.2) 368 : audit [INF] from='client.? 172.21.15.67:0/994064006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:08.765509+0000 mon.c (mon.2) 369 : audit [INF] from='client.? 172.21.15.67:0/2585205636' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:08.768760+0000 mon.a (mon.0) 1313 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:08.770016+0000 mon.a (mon.0) 1314 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:09.466552+0000 mon.c (mon.2) 370 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:09.467924+0000 mon.c (mon.2) 371 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:30:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:09 smithi067 bash[24488]: audit 2024-02-10T08:30:09.468773+0000 mon.a (mon.0) 1315 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:30:10.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-10T08:30:10.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [==========] Running 56 tests from 4 test suites. 2024-02-10T08:30:10.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-10T08:30:10.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio 2024-02-10T08:30:10.801 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (11180 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (23657 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6895 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2687 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3213 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3304 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3003 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3004 ms) 2024-02-10T08:30:10.802 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3014 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3016 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (2968 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3012 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3039 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3048 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3052 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3027 ms) 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-10T08:30:10.803 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3463 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2560 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (2806 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3053 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3018 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3004 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3034 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2900 ms) 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-10T08:30:10.804 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3473 ms) 2024-02-10T08:30:10.805 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-10T08:30:10.805 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2552 ms) 2024-02-10T08:30:10.805 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-10T08:30:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.761961+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.762226+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.762605+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]': finished 2024-02-10T08:30:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: cluster 2024-02-10T08:30:09.762712+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-10T08:30:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.766137+0000 mon.c (mon.2) 372 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.769028+0000 mon.c (mon.2) 373 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.772059+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.785935+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.811849+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.813167+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.813672+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi067-54237-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.824776+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: audit 2024-02-10T08:30:09.825279+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: cluster 2024-02-10T08:30:09.962163+0000 mgr.y (mgr.14547) 319 : cluster [DBG] pgmap v325: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[24488]: cluster 2024-02-10T08:30:10.715019+0000 mon.a (mon.0) 1325 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.761961+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.762226+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.762605+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]': finished 2024-02-10T08:30:11.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: cluster 2024-02-10T08:30:09.762712+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.766137+0000 mon.c (mon.2) 372 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.769028+0000 mon.c (mon.2) 373 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.772059+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.785935+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.811849+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.813167+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.813672+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi067-54237-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.824776+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: audit 2024-02-10T08:30:09.825279+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: cluster 2024-02-10T08:30:09.962163+0000 mgr.y (mgr.14547) 319 : cluster [DBG] pgmap v325: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:10 smithi067 bash[17385]: cluster 2024-02-10T08:30:10.715019+0000 mon.a (mon.0) 1325 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.761961+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi067-54237-28"}]': finished 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.762226+0000 mon.a (mon.0) 1317 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi067-54245-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.762605+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]': finished 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: cluster 2024-02-10T08:30:09.762712+0000 mon.a (mon.0) 1319 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.766137+0000 mon.c (mon.2) 372 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.769028+0000 mon.c (mon.2) 373 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.772059+0000 mon.c (mon.2) 374 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.785935+0000 mon.a (mon.0) 1320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.811849+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.813167+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.813672+0000 mon.a (mon.0) 1323 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi067-54237-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.824776+0000 mon.c (mon.2) 375 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: audit 2024-02-10T08:30:09.825279+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: cluster 2024-02-10T08:30:09.962163+0000 mgr.y (mgr.14547) 319 : cluster [DBG] pgmap v325: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:10 smithi134 bash[18722]: cluster 2024-02-10T08:30:10.715019+0000 mon.a (mon.0) 1325 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:30:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.791516+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T08:30:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.791697+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi067-54237-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.791807+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: cluster 2024-02-10T08:30:10.791923+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-10T08:30:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.795200+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi067-54237-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.797254+0000 mon.c (mon.2) 376 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.798662+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[17385]: audit 2024-02-10T08:30:10.799596+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.791516+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.791697+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi067-54237-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.791807+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: cluster 2024-02-10T08:30:10.791923+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.795200+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi067-54237-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.797254+0000 mon.c (mon.2) 376 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.798662+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:11 smithi067 bash[24488]: audit 2024-02-10T08:30:10.799596+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:12.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:30:12.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.791516+0000 mon.a (mon.0) 1326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T08:30:12.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.791697+0000 mon.a (mon.0) 1327 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi067-54237-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.791807+0000 mon.a (mon.0) 1328 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: cluster 2024-02-10T08:30:10.791923+0000 mon.a (mon.0) 1329 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-10T08:30:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.795200+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi067-54237-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.797254+0000 mon.c (mon.2) 376 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.798662+0000 mon.c (mon.2) 377 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:11 smithi134 bash[18722]: audit 2024-02-10T08:30:10.799596+0000 mon.a (mon.0) 1331 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]: dispatch 2024-02-10T08:30:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: cluster 2024-02-10T08:30:11.787783+0000 mon.a (mon.0) 1332 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: audit 2024-02-10T08:30:11.806448+0000 mon.c (mon.2) 378 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: audit 2024-02-10T08:30:11.934717+0000 mgr.y (mgr.14547) 320 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: cluster 2024-02-10T08:30:11.963084+0000 mgr.y (mgr.14547) 321 : cluster [DBG] pgmap v328: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:30:13.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: audit 2024-02-10T08:30:12.264008+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]': finished 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: cluster 2024-02-10T08:30:12.264160+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: audit 2024-02-10T08:30:12.265838+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.67:0/1916577096' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[24488]: audit 2024-02-10T08:30:12.271690+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: cluster 2024-02-10T08:30:11.787783+0000 mon.a (mon.0) 1332 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: audit 2024-02-10T08:30:11.806448+0000 mon.c (mon.2) 378 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: audit 2024-02-10T08:30:11.934717+0000 mgr.y (mgr.14547) 320 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: cluster 2024-02-10T08:30:11.963084+0000 mgr.y (mgr.14547) 321 : cluster [DBG] pgmap v328: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: audit 2024-02-10T08:30:12.264008+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]': finished 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: cluster 2024-02-10T08:30:12.264160+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: audit 2024-02-10T08:30:12.265838+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.67:0/1916577096' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:13.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:12 smithi067 bash[17385]: audit 2024-02-10T08:30:12.271690+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: cluster 2024-02-10T08:30:11.787783+0000 mon.a (mon.0) 1332 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:13.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: audit 2024-02-10T08:30:11.806448+0000 mon.c (mon.2) 378 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: audit 2024-02-10T08:30:11.934717+0000 mgr.y (mgr.14547) 320 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: cluster 2024-02-10T08:30:11.963084+0000 mgr.y (mgr.14547) 321 : cluster [DBG] pgmap v328: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: audit 2024-02-10T08:30:12.264008+0000 mon.a (mon.0) 1333 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-15"}]': finished 2024-02-10T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: cluster 2024-02-10T08:30:12.264160+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-10T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: audit 2024-02-10T08:30:12.265838+0000 mon.c (mon.2) 379 : audit [INF] from='client.? 172.21.15.67:0/1916577096' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:13.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:12 smithi134 bash[18722]: audit 2024-02-10T08:30:12.271690+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:30:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:30:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[17385]: audit 2024-02-10T08:30:12.809697+0000 mon.c (mon.2) 380 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[17385]: audit 2024-02-10T08:30:12.810415+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi067-54237-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[17385]: audit 2024-02-10T08:30:12.811104+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[17385]: cluster 2024-02-10T08:30:12.811371+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-10T08:30:14.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[24488]: audit 2024-02-10T08:30:12.809697+0000 mon.c (mon.2) 380 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[24488]: audit 2024-02-10T08:30:12.810415+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi067-54237-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[24488]: audit 2024-02-10T08:30:12.811104+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:14.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:13 smithi067 bash[24488]: cluster 2024-02-10T08:30:12.811371+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-10T08:30:14.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:13 smithi134 bash[18722]: audit 2024-02-10T08:30:12.809697+0000 mon.c (mon.2) 380 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:14.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:13 smithi134 bash[18722]: audit 2024-02-10T08:30:12.810415+0000 mon.a (mon.0) 1336 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi067-54237-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:14.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:13 smithi134 bash[18722]: audit 2024-02-10T08:30:12.811104+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi067-54245-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:14.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:13 smithi134 bash[18722]: cluster 2024-02-10T08:30:12.811371+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-10T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: cluster 2024-02-10T08:30:13.814299+0000 mon.a (mon.0) 1339 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-10T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:13.815783+0000 mon.c (mon.2) 381 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:15.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:13.817548+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:13.824464+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: cluster 2024-02-10T08:30:13.964048+0000 mgr.y (mgr.14547) 322 : cluster [DBG] pgmap v331: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:14.817139+0000 mon.c (mon.2) 383 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:14.818042+0000 mon.a (mon.0) 1341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: cluster 2024-02-10T08:30:14.818512+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:14.822251+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[17385]: audit 2024-02-10T08:30:14.824162+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.67:0/2024003349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: cluster 2024-02-10T08:30:13.814299+0000 mon.a (mon.0) 1339 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:13.815783+0000 mon.c (mon.2) 381 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:13.817548+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:13.824464+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: cluster 2024-02-10T08:30:13.964048+0000 mgr.y (mgr.14547) 322 : cluster [DBG] pgmap v331: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:15.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:14.817139+0000 mon.c (mon.2) 383 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:15.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:14.818042+0000 mon.a (mon.0) 1341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:15.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: cluster 2024-02-10T08:30:14.818512+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-10T08:30:15.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:14.822251+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:15.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:14 smithi067 bash[24488]: audit 2024-02-10T08:30:14.824162+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.67:0/2024003349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: cluster 2024-02-10T08:30:13.814299+0000 mon.a (mon.0) 1339 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:13.815783+0000 mon.c (mon.2) 381 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:13.817548+0000 mon.c (mon.2) 382 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:13.824464+0000 mon.a (mon.0) 1340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: cluster 2024-02-10T08:30:13.964048+0000 mgr.y (mgr.14547) 322 : cluster [DBG] pgmap v331: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:14.817139+0000 mon.c (mon.2) 383 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:14.818042+0000 mon.a (mon.0) 1341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: cluster 2024-02-10T08:30:14.818512+0000 mon.a (mon.0) 1342 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:14.822251+0000 mon.a (mon.0) 1343 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:15.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:14 smithi134 bash[18722]: audit 2024-02-10T08:30:14.824162+0000 mon.b (mon.1) 65 : audit [INF] from='client.? 172.21.15.67:0/2024003349' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:16.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:14.828791+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:14.852930+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:14.853589+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.729200+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.729474+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.729773+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: cluster 2024-02-10T08:30:15.729879+0000 mon.a (mon.0) 1349 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.731572+0000 mon.a (mon.0) 1350 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.731895+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.744583+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.773419+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[17385]: audit 2024-02-10T08:30:15.818646+0000 mon.c (mon.2) 386 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:14.828791+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:16.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:14.852930+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:14.853589+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.729200+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.729474+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.729773+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: cluster 2024-02-10T08:30:15.729879+0000 mon.a (mon.0) 1349 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.731572+0000 mon.a (mon.0) 1350 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.731895+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.744583+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.773419+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:16.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:15 smithi067 bash[24488]: audit 2024-02-10T08:30:15.818646+0000 mon.c (mon.2) 386 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:14.828791+0000 mon.a (mon.0) 1344 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:14.852930+0000 mon.c (mon.2) 384 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:14.853589+0000 mon.a (mon.0) 1345 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.729200+0000 mon.a (mon.0) 1346 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:16.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.729474+0000 mon.a (mon.0) 1347 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi067-54245-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.729773+0000 mon.a (mon.0) 1348 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: cluster 2024-02-10T08:30:15.729879+0000 mon.a (mon.0) 1349 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.731572+0000 mon.a (mon.0) 1350 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]: dispatch 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.731895+0000 mon.c (mon.2) 385 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.744583+0000 mon.a (mon.0) 1351 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.773419+0000 mon.a (mon.0) 1352 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:16.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:15 smithi134 bash[18722]: audit 2024-02-10T08:30:15.818646+0000 mon.c (mon.2) 386 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: cluster 2024-02-10T08:30:15.964968+0000 mgr.y (mgr.14547) 323 : cluster [DBG] pgmap v334: 292 pgs: 64 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:16.729642+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:16.729742+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]': finished 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: cluster 2024-02-10T08:30:16.729867+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:16.733182+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]: dispatch 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:16.739377+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]: dispatch 2024-02-10T08:30:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:16.763436+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:16.820215+0000 mon.c (mon.2) 389 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:18.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:17.233413+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:17.239426+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:17.239959+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:17.240698+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:18.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:17 smithi134 bash[18722]: audit 2024-02-10T08:30:17.241191+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:17] "GET /metrics HTTP/1.1" 200 190391 "" "Prometheus/2.33.4" 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: cluster 2024-02-10T08:30:15.964968+0000 mgr.y (mgr.14547) 323 : cluster [DBG] pgmap v334: 292 pgs: 64 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:16.729642+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:16.729742+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]': finished 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: cluster 2024-02-10T08:30:16.729867+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:16.733182+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:16.739377+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:16.763436+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:16.820215+0000 mon.c (mon.2) 389 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:17.233413+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:17.239426+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:17.239959+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:17.240698+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[17385]: audit 2024-02-10T08:30:17.241191+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: cluster 2024-02-10T08:30:15.964968+0000 mgr.y (mgr.14547) 323 : cluster [DBG] pgmap v334: 292 pgs: 64 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:16.729642+0000 mon.a (mon.0) 1353 : audit [INF] from='client.? 172.21.15.67:0/2961568121' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi067-54237-29"}]': finished 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:16.729742+0000 mon.a (mon.0) 1354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-17"}]': finished 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: cluster 2024-02-10T08:30:16.729867+0000 mon.a (mon.0) 1355 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:16.733182+0000 mon.c (mon.2) 387 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:16.739377+0000 mon.a (mon.0) 1356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:16.763436+0000 mon.c (mon.2) 388 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:16.820215+0000 mon.c (mon.2) 389 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:17.233413+0000 mon.a (mon.0) 1357 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:17.239426+0000 mon.c (mon.2) 390 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:17.239959+0000 mon.a (mon.0) 1358 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:17.240698+0000 mon.c (mon.2) 391 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:17 smithi067 bash[24488]: audit 2024-02-10T08:30:17.241191+0000 mon.a (mon.0) 1359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: cluster 2024-02-10T08:30:17.724822+0000 mon.a (mon.0) 1360 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.737744+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]': finished 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.737948+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: cluster 2024-02-10T08:30:17.739244+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.743047+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.756401+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? 172.21.15.67:0/2914009189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi067-54245-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.756893+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.821290+0000 mon.c (mon.2) 393 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.833363+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:19 smithi134 bash[18722]: audit 2024-02-10T08:30:17.833922+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: cluster 2024-02-10T08:30:17.724822+0000 mon.a (mon.0) 1360 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.737744+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]': finished 2024-02-10T08:30:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.737948+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: cluster 2024-02-10T08:30:17.739244+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-10T08:30:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.743047+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.756401+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? 172.21.15.67:0/2914009189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi067-54245-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.756893+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.821290+0000 mon.c (mon.2) 393 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.833363+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[24488]: audit 2024-02-10T08:30:17.833922+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: cluster 2024-02-10T08:30:17.724822+0000 mon.a (mon.0) 1360 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.737744+0000 mon.a (mon.0) 1361 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-17", "mode": "writeback"}]': finished 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.737948+0000 mon.a (mon.0) 1362 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi067-54237-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: cluster 2024-02-10T08:30:17.739244+0000 mon.a (mon.0) 1363 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-10T08:30:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.743047+0000 mon.c (mon.2) 392 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:19.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.756401+0000 mon.a (mon.0) 1364 : audit [INF] from='client.? 172.21.15.67:0/2914009189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi067-54245-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:19.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.756893+0000 mon.a (mon.0) 1365 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:19.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.821290+0000 mon.c (mon.2) 393 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:19.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.833363+0000 mon.c (mon.2) 394 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:19.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:19 smithi067 bash[17385]: audit 2024-02-10T08:30:17.833922+0000 mon.a (mon.0) 1366 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: cluster 2024-02-10T08:30:17.965966+0000 mgr.y (mgr.14547) 324 : cluster [DBG] pgmap v337: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:18.739956+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.67:0/2914009189' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi067-54245-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:18.822821+0000 mon.c (mon.2) 395 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:19.194735+0000 mon.a (mon.0) 1368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: cluster 2024-02-10T08:30:19.194985+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:19.196145+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:19.207083+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: cluster 2024-02-10T08:30:19.733991+0000 mon.a (mon.0) 1371 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:19.743911+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: audit 2024-02-10T08:30:19.744047+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]': finished 2024-02-10T08:30:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:20 smithi134 bash[18722]: cluster 2024-02-10T08:30:19.744182+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-10T08:30:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: cluster 2024-02-10T08:30:17.965966+0000 mgr.y (mgr.14547) 324 : cluster [DBG] pgmap v337: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:18.739956+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.67:0/2914009189' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi067-54245-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:18.822821+0000 mon.c (mon.2) 395 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:19.194735+0000 mon.a (mon.0) 1368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: cluster 2024-02-10T08:30:19.194985+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:19.196145+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:19.207083+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: cluster 2024-02-10T08:30:19.733991+0000 mon.a (mon.0) 1371 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:19.743911+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: audit 2024-02-10T08:30:19.744047+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]': finished 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[17385]: cluster 2024-02-10T08:30:19.744182+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: cluster 2024-02-10T08:30:17.965966+0000 mgr.y (mgr.14547) 324 : cluster [DBG] pgmap v337: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:18.739956+0000 mon.a (mon.0) 1367 : audit [INF] from='client.? 172.21.15.67:0/2914009189' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi067-54245-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:18.822821+0000 mon.c (mon.2) 395 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:19.194735+0000 mon.a (mon.0) 1368 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: cluster 2024-02-10T08:30:19.194985+0000 mon.a (mon.0) 1369 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-10T08:30:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:19.196145+0000 mon.c (mon.2) 396 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:19.207083+0000 mon.a (mon.0) 1370 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]: dispatch 2024-02-10T08:30:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: cluster 2024-02-10T08:30:19.733991+0000 mon.a (mon.0) 1371 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:19.743911+0000 mon.a (mon.0) 1372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi067-54237-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: audit 2024-02-10T08:30:19.744047+0000 mon.a (mon.0) 1373 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-17"}]': finished 2024-02-10T08:30:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:20 smithi067 bash[24488]: cluster 2024-02-10T08:30:19.744182+0000 mon.a (mon.0) 1374 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-10T08:30:21.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[18722]: audit 2024-02-10T08:30:19.824457+0000 mon.c (mon.2) 397 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[18722]: cluster 2024-02-10T08:30:19.966632+0000 mgr.y (mgr.14547) 325 : cluster [DBG] pgmap v340: 268 pgs: 8 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[18722]: cluster 2024-02-10T08:30:20.753667+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-10T08:30:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[18722]: audit 2024-02-10T08:30:20.765591+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.67:0/685349838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[18722]: audit 2024-02-10T08:30:20.826074+0000 mon.c (mon.2) 399 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[18722]: audit 2024-02-10T08:30:21.224315+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[24488]: audit 2024-02-10T08:30:19.824457+0000 mon.c (mon.2) 397 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[24488]: cluster 2024-02-10T08:30:19.966632+0000 mgr.y (mgr.14547) 325 : cluster [DBG] pgmap v340: 268 pgs: 8 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[24488]: cluster 2024-02-10T08:30:20.753667+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-10T08:30:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[24488]: audit 2024-02-10T08:30:20.765591+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.67:0/685349838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[24488]: audit 2024-02-10T08:30:20.826074+0000 mon.c (mon.2) 399 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[24488]: audit 2024-02-10T08:30:21.224315+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[17385]: audit 2024-02-10T08:30:19.824457+0000 mon.c (mon.2) 397 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[17385]: cluster 2024-02-10T08:30:19.966632+0000 mgr.y (mgr.14547) 325 : cluster [DBG] pgmap v340: 268 pgs: 8 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[17385]: cluster 2024-02-10T08:30:20.753667+0000 mon.a (mon.0) 1375 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-10T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[17385]: audit 2024-02-10T08:30:20.765591+0000 mon.c (mon.2) 398 : audit [INF] from='client.? 172.21.15.67:0/685349838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[17385]: audit 2024-02-10T08:30:20.826074+0000 mon.c (mon.2) 399 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:21 smithi067 bash[17385]: audit 2024-02-10T08:30:21.224315+0000 mon.a (mon.0) 1376 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:22.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:30:21 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:30:23.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: audit 2024-02-10T08:30:21.760862+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: cluster 2024-02-10T08:30:21.760999+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-10T08:30:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: audit 2024-02-10T08:30:21.775008+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: audit 2024-02-10T08:30:21.775718+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: audit 2024-02-10T08:30:21.776242+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: audit 2024-02-10T08:30:21.777246+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:22 smithi134 bash[18722]: audit 2024-02-10T08:30:21.827622+0000 mon.c (mon.2) 402 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:23.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: audit 2024-02-10T08:30:21.760862+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:23.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: cluster 2024-02-10T08:30:21.760999+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-10T08:30:23.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: audit 2024-02-10T08:30:21.775008+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: audit 2024-02-10T08:30:21.775718+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: audit 2024-02-10T08:30:21.776242+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: audit 2024-02-10T08:30:21.777246+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[24488]: audit 2024-02-10T08:30:21.827622+0000 mon.c (mon.2) 402 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: audit 2024-02-10T08:30:21.760862+0000 mon.a (mon.0) 1377 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi067-54245-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: cluster 2024-02-10T08:30:21.760999+0000 mon.a (mon.0) 1378 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: audit 2024-02-10T08:30:21.775008+0000 mon.c (mon.2) 400 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: audit 2024-02-10T08:30:21.775718+0000 mon.a (mon.0) 1379 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: audit 2024-02-10T08:30:21.776242+0000 mon.c (mon.2) 401 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: audit 2024-02-10T08:30:21.777246+0000 mon.a (mon.0) 1380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:22 smithi067 bash[17385]: audit 2024-02-10T08:30:21.827622+0000 mon.c (mon.2) 402 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:30:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:21.946221+0000 mgr.y (mgr.14547) 326 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: cluster 2024-02-10T08:30:21.967480+0000 mgr.y (mgr.14547) 327 : cluster [DBG] pgmap v343: 292 pgs: 32 creating+peering, 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.761384+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.761565+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: cluster 2024-02-10T08:30:22.761772+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.770128+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.786139+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.790530+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.791310+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[24488]: audit 2024-02-10T08:30:22.829140+0000 mon.c (mon.2) 405 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:21.946221+0000 mgr.y (mgr.14547) 326 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: cluster 2024-02-10T08:30:21.967480+0000 mgr.y (mgr.14547) 327 : cluster [DBG] pgmap v343: 292 pgs: 32 creating+peering, 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.761384+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.761565+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: cluster 2024-02-10T08:30:22.761772+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-10T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.770128+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.786139+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.790530+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.791310+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:23 smithi067 bash[17385]: audit 2024-02-10T08:30:22.829140+0000 mon.c (mon.2) 405 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:21.946221+0000 mgr.y (mgr.14547) 326 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: cluster 2024-02-10T08:30:21.967480+0000 mgr.y (mgr.14547) 327 : cluster [DBG] pgmap v343: 292 pgs: 32 creating+peering, 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.761384+0000 mon.a (mon.0) 1381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.761565+0000 mon.a (mon.0) 1382 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: cluster 2024-02-10T08:30:22.761772+0000 mon.a (mon.0) 1383 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.770128+0000 mon.c (mon.2) 403 : audit [INF] from='client.? 172.21.15.67:0/2070051441' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.786139+0000 mon.a (mon.0) 1384 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]: dispatch 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.790530+0000 mon.c (mon.2) 404 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.791310+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:23 smithi134 bash[18722]: audit 2024-02-10T08:30:22.829140+0000 mon.c (mon.2) 405 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.787045+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.787240+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: cluster 2024-02-10T08:30:23.787417+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.791358+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.791768+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.67:0/719025105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.813901+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.814285+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.818940+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.819348+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.820020+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.820413+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.821037+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.821446+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:23.830745+0000 mon.c (mon.2) 411 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.787045+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.787240+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: cluster 2024-02-10T08:30:23.787417+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.791358+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.791768+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.67:0/719025105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.813901+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.814285+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.818940+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.819348+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.820020+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.820413+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.821037+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.821446+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:23.830745+0000 mon.c (mon.2) 411 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: cluster 2024-02-10T08:30:23.968402+0000 mgr.y (mgr.14547) 328 : cluster [DBG] pgmap v346: 292 pgs: 64 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.788900+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]': finished 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.789068+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:25.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.789296+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: cluster 2024-02-10T08:30:24.789403+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.793714+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]: dispatch 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.794384+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.796266+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]: dispatch 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[24488]: audit 2024-02-10T08:30:24.796756+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: cluster 2024-02-10T08:30:23.968402+0000 mgr.y (mgr.14547) 328 : cluster [DBG] pgmap v346: 292 pgs: 64 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.788900+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]': finished 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.789068+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.789296+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: cluster 2024-02-10T08:30:24.789403+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-10T08:30:25.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.793714+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]: dispatch 2024-02-10T08:30:25.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.794384+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.796266+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]: dispatch 2024-02-10T08:30:25.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:24 smithi067 bash[17385]: audit 2024-02-10T08:30:24.796756+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.787045+0000 mon.a (mon.0) 1386 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi067-54237-30"}]': finished 2024-02-10T08:30:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.787240+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: cluster 2024-02-10T08:30:23.787417+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-10T08:30:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.791358+0000 mon.c (mon.2) 406 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.791768+0000 mon.c (mon.2) 407 : audit [INF] from='client.? 172.21.15.67:0/719025105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.813901+0000 mon.a (mon.0) 1389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.814285+0000 mon.a (mon.0) 1390 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.818940+0000 mon.c (mon.2) 408 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.819348+0000 mon.a (mon.0) 1391 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.820020+0000 mon.c (mon.2) 409 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.820413+0000 mon.a (mon.0) 1392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.821037+0000 mon.c (mon.2) 410 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.821446+0000 mon.a (mon.0) 1393 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:23.830745+0000 mon.c (mon.2) 411 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: cluster 2024-02-10T08:30:23.968402+0000 mgr.y (mgr.14547) 328 : cluster [DBG] pgmap v346: 292 pgs: 64 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.788900+0000 mon.a (mon.0) 1394 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-19"}]': finished 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.789068+0000 mon.a (mon.0) 1395 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi067-54245-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.789296+0000 mon.a (mon.0) 1396 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi067-54237-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: cluster 2024-02-10T08:30:24.789403+0000 mon.a (mon.0) 1397 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-10T08:30:25.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.793714+0000 mon.c (mon.2) 412 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]: dispatch 2024-02-10T08:30:25.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.794384+0000 mon.c (mon.2) 413 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:25.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.796266+0000 mon.a (mon.0) 1398 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]: dispatch 2024-02-10T08:30:25.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:24 smithi134 bash[18722]: audit 2024-02-10T08:30:24.796756+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:26 smithi134 bash[18722]: audit 2024-02-10T08:30:24.832081+0000 mon.c (mon.2) 414 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:26 smithi134 bash[18722]: cluster 2024-02-10T08:30:25.782982+0000 mon.a (mon.0) 1400 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:26 smithi134 bash[18722]: audit 2024-02-10T08:30:25.791016+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]': finished 2024-02-10T08:30:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:26 smithi067 bash[17385]: audit 2024-02-10T08:30:24.832081+0000 mon.c (mon.2) 414 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:26 smithi067 bash[17385]: cluster 2024-02-10T08:30:25.782982+0000 mon.a (mon.0) 1400 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:26 smithi067 bash[17385]: audit 2024-02-10T08:30:25.791016+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]': finished 2024-02-10T08:30:26.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:26 smithi067 bash[24488]: audit 2024-02-10T08:30:24.832081+0000 mon.c (mon.2) 414 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:26.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:26 smithi067 bash[24488]: cluster 2024-02-10T08:30:25.782982+0000 mon.a (mon.0) 1400 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:26.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:26 smithi067 bash[24488]: audit 2024-02-10T08:30:25.791016+0000 mon.a (mon.0) 1401 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-19", "mode": "writeback"}]': finished 2024-02-10T08:30:27.532 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:25.833400+0000 mon.c (mon.2) 415 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:27.533 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: cluster 2024-02-10T08:30:25.969194+0000 mgr.y (mgr.14547) 329 : cluster [DBG] pgmap v349: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:27.533 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.043817+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:27.533 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: cluster 2024-02-10T08:30:26.249396+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-10T08:30:27.534 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.259906+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:27.534 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.799694+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:27.534 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.799890+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:27.534 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: cluster 2024-02-10T08:30:26.800035+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-10T08:30:27.534 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.801774+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:27.535 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.802742+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:27.535 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.805613+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.67:0/1345429888' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:27.535 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.812671+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:27.535 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:27 smithi134 bash[18722]: audit 2024-02-10T08:30:26.834956+0000 mon.c (mon.2) 419 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:25.833400+0000 mon.c (mon.2) 415 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: cluster 2024-02-10T08:30:25.969194+0000 mgr.y (mgr.14547) 329 : cluster [DBG] pgmap v349: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.043817+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: cluster 2024-02-10T08:30:26.249396+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.259906+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.799694+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.799890+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: cluster 2024-02-10T08:30:26.800035+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.801774+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.802742+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.805613+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.67:0/1345429888' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.812671+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17385]: audit 2024-02-10T08:30:26.834956+0000 mon.c (mon.2) 419 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:25.833400+0000 mon.c (mon.2) 415 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: cluster 2024-02-10T08:30:25.969194+0000 mgr.y (mgr.14547) 329 : cluster [DBG] pgmap v349: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.043817+0000 mon.c (mon.2) 416 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: cluster 2024-02-10T08:30:26.249396+0000 mon.a (mon.0) 1402 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.259906+0000 mon.a (mon.0) 1403 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.799694+0000 mon.a (mon.0) 1404 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi067-54237-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.799890+0000 mon.a (mon.0) 1405 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:27.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: cluster 2024-02-10T08:30:26.800035+0000 mon.a (mon.0) 1406 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-10T08:30:27.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.801774+0000 mon.c (mon.2) 417 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:27.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.802742+0000 mon.a (mon.0) 1407 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]: dispatch 2024-02-10T08:30:27.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.805613+0000 mon.c (mon.2) 418 : audit [INF] from='client.? 172.21.15.67:0/1345429888' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:27.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.812671+0000 mon.a (mon.0) 1408 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:27.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[24488]: audit 2024-02-10T08:30:26.834956+0000 mon.c (mon.2) 419 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:30:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:27] "GET /metrics HTTP/1.1" 200 190391 "" "Prometheus/2.33.4" 2024-02-10T08:30:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[17385]: cluster 2024-02-10T08:30:27.789968+0000 mon.a (mon.0) 1409 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[17385]: audit 2024-02-10T08:30:27.800535+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]': finished 2024-02-10T08:30:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[17385]: audit 2024-02-10T08:30:27.800774+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[17385]: cluster 2024-02-10T08:30:27.801046+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[17385]: audit 2024-02-10T08:30:27.836288+0000 mon.c (mon.2) 420 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[24488]: cluster 2024-02-10T08:30:27.789968+0000 mon.a (mon.0) 1409 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[24488]: audit 2024-02-10T08:30:27.800535+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]': finished 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[24488]: audit 2024-02-10T08:30:27.800774+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[24488]: cluster 2024-02-10T08:30:27.801046+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-10T08:30:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:28 smithi067 bash[24488]: audit 2024-02-10T08:30:27.836288+0000 mon.c (mon.2) 420 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:28 smithi134 bash[18722]: cluster 2024-02-10T08:30:27.789968+0000 mon.a (mon.0) 1409 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:28 smithi134 bash[18722]: audit 2024-02-10T08:30:27.800535+0000 mon.a (mon.0) 1410 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-19"}]': finished 2024-02-10T08:30:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:28 smithi134 bash[18722]: audit 2024-02-10T08:30:27.800774+0000 mon.a (mon.0) 1411 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:28 smithi134 bash[18722]: cluster 2024-02-10T08:30:27.801046+0000 mon.a (mon.0) 1412 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-10T08:30:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:28 smithi134 bash[18722]: audit 2024-02-10T08:30:27.836288+0000 mon.c (mon.2) 420 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: cluster 2024-02-10T08:30:27.970016+0000 mgr.y (mgr.14547) 330 : cluster [DBG] pgmap v352: 300 pgs: 40 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:30:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: cluster 2024-02-10T08:30:28.807260+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: audit 2024-02-10T08:30:28.822571+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.67:0/748144360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: audit 2024-02-10T08:30:28.823161+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: audit 2024-02-10T08:30:28.823728+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: audit 2024-02-10T08:30:28.837398+0000 mon.c (mon.2) 423 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[17385]: audit 2024-02-10T08:30:29.285417+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: cluster 2024-02-10T08:30:27.970016+0000 mgr.y (mgr.14547) 330 : cluster [DBG] pgmap v352: 300 pgs: 40 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: cluster 2024-02-10T08:30:28.807260+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: audit 2024-02-10T08:30:28.822571+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.67:0/748144360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: audit 2024-02-10T08:30:28.823161+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: audit 2024-02-10T08:30:28.823728+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: audit 2024-02-10T08:30:28.837398+0000 mon.c (mon.2) 423 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:29 smithi067 bash[24488]: audit 2024-02-10T08:30:29.285417+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: cluster 2024-02-10T08:30:27.970016+0000 mgr.y (mgr.14547) 330 : cluster [DBG] pgmap v352: 300 pgs: 40 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:30:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: cluster 2024-02-10T08:30:28.807260+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-10T08:30:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: audit 2024-02-10T08:30:28.822571+0000 mon.c (mon.2) 421 : audit [INF] from='client.? 172.21.15.67:0/748144360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: audit 2024-02-10T08:30:28.823161+0000 mon.c (mon.2) 422 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: audit 2024-02-10T08:30:28.823728+0000 mon.a (mon.0) 1414 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: audit 2024-02-10T08:30:28.837398+0000 mon.c (mon.2) 423 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:29 smithi134 bash[18722]: audit 2024-02-10T08:30:29.285417+0000 mon.a (mon.0) 1415 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.821191+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.821366+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: cluster 2024-02-10T08:30:29.821574+0000 mon.a (mon.0) 1418 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.823993+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.824517+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.825068+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.827179+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: audit 2024-02-10T08:30:29.838334+0000 mon.c (mon.2) 426 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[17385]: cluster 2024-02-10T08:30:29.970925+0000 mgr.y (mgr.14547) 331 : cluster [DBG] pgmap v355: 324 pgs: 96 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.821191+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.821366+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: cluster 2024-02-10T08:30:29.821574+0000 mon.a (mon.0) 1418 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.823993+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.824517+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.825068+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.827179+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: audit 2024-02-10T08:30:29.838334+0000 mon.c (mon.2) 426 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:31.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:30 smithi067 bash[24488]: cluster 2024-02-10T08:30:29.970925+0000 mgr.y (mgr.14547) 331 : cluster [DBG] pgmap v355: 324 pgs: 96 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.821191+0000 mon.a (mon.0) 1416 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.821366+0000 mon.a (mon.0) 1417 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: cluster 2024-02-10T08:30:29.821574+0000 mon.a (mon.0) 1418 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.823993+0000 mon.c (mon.2) 424 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.824517+0000 mon.c (mon.2) 425 : audit [INF] from='client.? 172.21.15.67:0/1653776894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.825068+0000 mon.a (mon.0) 1419 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.827179+0000 mon.a (mon.0) 1420 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]: dispatch 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: audit 2024-02-10T08:30:29.838334+0000 mon.c (mon.2) 426 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:30 smithi134 bash[18722]: cluster 2024-02-10T08:30:29.970925+0000 mgr.y (mgr.14547) 331 : cluster [DBG] pgmap v355: 324 pgs: 96 unknown, 228 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.824252+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.825507+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: cluster 2024-02-10T08:30:30.829177+0000 mon.a (mon.0) 1423 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.835997+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.67:0/2367795538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.843937+0000 mon.c (mon.2) 428 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.855465+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.856727+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.859397+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.863174+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.863246+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.864290+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.864376+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.865387+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:30.865471+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.824252+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.825507+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: cluster 2024-02-10T08:30:30.829177+0000 mon.a (mon.0) 1423 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.835997+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.67:0/2367795538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.843937+0000 mon.c (mon.2) 428 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.855465+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.856727+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.859397+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.863174+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.863246+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.864290+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.864376+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.865387+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:30.865471+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.823929+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.824141+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.824291+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: cluster 2024-02-10T08:30:31.824484+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.827596+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.828406+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.829908+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:32.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[17385]: audit 2024-02-10T08:30:31.830330+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.823929+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.824141+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.824291+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: cluster 2024-02-10T08:30:31.824484+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.827596+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.828406+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.829908+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:32.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:31 smithi067 bash[24488]: audit 2024-02-10T08:30:31.830330+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.824252+0000 mon.a (mon.0) 1421 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.825507+0000 mon.a (mon.0) 1422 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi067-54237-31"}]': finished 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: cluster 2024-02-10T08:30:30.829177+0000 mon.a (mon.0) 1423 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.835997+0000 mon.c (mon.2) 427 : audit [INF] from='client.? 172.21.15.67:0/2367795538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.843937+0000 mon.c (mon.2) 428 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.855465+0000 mon.c (mon.2) 429 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.856727+0000 mon.a (mon.0) 1424 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.859397+0000 mon.a (mon.0) 1425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.863174+0000 mon.a (mon.0) 1426 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.863246+0000 mon.b (mon.1) 66 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.864290+0000 mon.a (mon.0) 1427 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.864376+0000 mon.b (mon.1) 67 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.865387+0000 mon.a (mon.0) 1428 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:30.865471+0000 mon.b (mon.1) 68 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.823929+0000 mon.a (mon.0) 1429 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi067-54245-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.824141+0000 mon.a (mon.0) 1430 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.824291+0000 mon.a (mon.0) 1431 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi067-54237-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: cluster 2024-02-10T08:30:31.824484+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.827596+0000 mon.c (mon.2) 430 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.828406+0000 mon.b (mon.1) 69 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.829908+0000 mon.a (mon.0) 1433 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:32.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:31 smithi134 bash[18722]: audit 2024-02-10T08:30:31.830330+0000 mon.a (mon.0) 1434 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:30:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:30:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:33 smithi134 bash[18722]: audit 2024-02-10T08:30:31.847219+0000 mon.c (mon.2) 431 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:33 smithi134 bash[18722]: audit 2024-02-10T08:30:31.957691+0000 mgr.y (mgr.14547) 332 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:33 smithi134 bash[18722]: cluster 2024-02-10T08:30:31.971809+0000 mgr.y (mgr.14547) 333 : cluster [DBG] pgmap v358: 356 pgs: 32 creating+peering, 324 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:30:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:33 smithi134 bash[18722]: audit 2024-02-10T08:30:32.827739+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]': finished 2024-02-10T08:30:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[17385]: audit 2024-02-10T08:30:31.847219+0000 mon.c (mon.2) 431 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[17385]: audit 2024-02-10T08:30:31.957691+0000 mgr.y (mgr.14547) 332 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[17385]: cluster 2024-02-10T08:30:31.971809+0000 mgr.y (mgr.14547) 333 : cluster [DBG] pgmap v358: 356 pgs: 32 creating+peering, 324 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:30:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[17385]: audit 2024-02-10T08:30:32.827739+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]': finished 2024-02-10T08:30:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[24488]: audit 2024-02-10T08:30:31.847219+0000 mon.c (mon.2) 431 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[24488]: audit 2024-02-10T08:30:31.957691+0000 mgr.y (mgr.14547) 332 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[24488]: cluster 2024-02-10T08:30:31.971809+0000 mgr.y (mgr.14547) 333 : cluster [DBG] pgmap v358: 356 pgs: 32 creating+peering, 324 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:30:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:33 smithi067 bash[24488]: audit 2024-02-10T08:30:32.827739+0000 mon.a (mon.0) 1435 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-21"}]': finished 2024-02-10T08:30:34.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:32.833093+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]: dispatch 2024-02-10T08:30:34.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:32.848392+0000 mon.c (mon.2) 433 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:34.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: cluster 2024-02-10T08:30:33.275757+0000 mon.a (mon.0) 1436 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:33.278041+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]: dispatch 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: cluster 2024-02-10T08:30:33.821947+0000 mon.a (mon.0) 1438 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:33.831305+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:33.831490+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]': finished 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: cluster 2024-02-10T08:30:33.831658+0000 mon.a (mon.0) 1441 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:33.851638+0000 mon.c (mon.2) 434 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:33.909168+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[24488]: audit 2024-02-10T08:30:33.909696+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:32.833093+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]: dispatch 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:32.848392+0000 mon.c (mon.2) 433 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: cluster 2024-02-10T08:30:33.275757+0000 mon.a (mon.0) 1436 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:33.278041+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]: dispatch 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: cluster 2024-02-10T08:30:33.821947+0000 mon.a (mon.0) 1438 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:33.831305+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:33.831490+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]': finished 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: cluster 2024-02-10T08:30:33.831658+0000 mon.a (mon.0) 1441 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:33.851638+0000 mon.c (mon.2) 434 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:33.909168+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:34.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:34 smithi067 bash[17385]: audit 2024-02-10T08:30:33.909696+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:32.833093+0000 mon.c (mon.2) 432 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]: dispatch 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:32.848392+0000 mon.c (mon.2) 433 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: cluster 2024-02-10T08:30:33.275757+0000 mon.a (mon.0) 1436 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:33.278041+0000 mon.a (mon.0) 1437 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]: dispatch 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: cluster 2024-02-10T08:30:33.821947+0000 mon.a (mon.0) 1438 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:33.831305+0000 mon.a (mon.0) 1439 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi067-54237-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:33.831490+0000 mon.a (mon.0) 1440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-21", "mode": "writeback"}]': finished 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: cluster 2024-02-10T08:30:33.831658+0000 mon.a (mon.0) 1441 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:33.851638+0000 mon.c (mon.2) 434 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:33.909168+0000 mon.c (mon.2) 435 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:34 smithi134 bash[18722]: audit 2024-02-10T08:30:33.909696+0000 mon.a (mon.0) 1442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[17385]: cluster 2024-02-10T08:30:33.972673+0000 mgr.y (mgr.14547) 334 : cluster [DBG] pgmap v361: 300 pgs: 8 unknown, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[17385]: audit 2024-02-10T08:30:34.832880+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[17385]: cluster 2024-02-10T08:30:34.832992+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[17385]: audit 2024-02-10T08:30:34.839919+0000 mon.c (mon.2) 436 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[17385]: audit 2024-02-10T08:30:34.854334+0000 mon.c (mon.2) 437 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[17385]: audit 2024-02-10T08:30:34.855191+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[24488]: cluster 2024-02-10T08:30:33.972673+0000 mgr.y (mgr.14547) 334 : cluster [DBG] pgmap v361: 300 pgs: 8 unknown, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[24488]: audit 2024-02-10T08:30:34.832880+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[24488]: cluster 2024-02-10T08:30:34.832992+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[24488]: audit 2024-02-10T08:30:34.839919+0000 mon.c (mon.2) 436 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[24488]: audit 2024-02-10T08:30:34.854334+0000 mon.c (mon.2) 437 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:35 smithi067 bash[24488]: audit 2024-02-10T08:30:34.855191+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:35 smithi134 bash[18722]: cluster 2024-02-10T08:30:33.972673+0000 mgr.y (mgr.14547) 334 : cluster [DBG] pgmap v361: 300 pgs: 8 unknown, 292 active+clean; 8.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:35 smithi134 bash[18722]: audit 2024-02-10T08:30:34.832880+0000 mon.a (mon.0) 1443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:35 smithi134 bash[18722]: cluster 2024-02-10T08:30:34.832992+0000 mon.a (mon.0) 1444 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-10T08:30:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:35 smithi134 bash[18722]: audit 2024-02-10T08:30:34.839919+0000 mon.c (mon.2) 436 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:35 smithi134 bash[18722]: audit 2024-02-10T08:30:34.854334+0000 mon.c (mon.2) 437 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:35 smithi134 bash[18722]: audit 2024-02-10T08:30:34.855191+0000 mon.a (mon.0) 1445 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]: dispatch 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: cluster 2024-02-10T08:30:35.848717+0000 mon.a (mon.0) 1446 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: audit 2024-02-10T08:30:35.858149+0000 mon.c (mon.2) 438 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: audit 2024-02-10T08:30:35.860435+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]': finished 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: cluster 2024-02-10T08:30:35.860822+0000 mon.a (mon.0) 1448 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: audit 2024-02-10T08:30:35.864251+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? 172.21.15.67:0/3725637981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: audit 2024-02-10T08:30:35.869399+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:36 smithi134 bash[18722]: audit 2024-02-10T08:30:35.869514+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: cluster 2024-02-10T08:30:35.848717+0000 mon.a (mon.0) 1446 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: audit 2024-02-10T08:30:35.858149+0000 mon.c (mon.2) 438 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: audit 2024-02-10T08:30:35.860435+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]': finished 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: cluster 2024-02-10T08:30:35.860822+0000 mon.a (mon.0) 1448 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: audit 2024-02-10T08:30:35.864251+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? 172.21.15.67:0/3725637981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: audit 2024-02-10T08:30:35.869399+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[17385]: audit 2024-02-10T08:30:35.869514+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: cluster 2024-02-10T08:30:35.848717+0000 mon.a (mon.0) 1446 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: audit 2024-02-10T08:30:35.858149+0000 mon.c (mon.2) 438 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: audit 2024-02-10T08:30:35.860435+0000 mon.a (mon.0) 1447 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-21"}]': finished 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: cluster 2024-02-10T08:30:35.860822+0000 mon.a (mon.0) 1448 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: audit 2024-02-10T08:30:35.864251+0000 mon.a (mon.0) 1449 : audit [INF] from='client.? 172.21.15.67:0/3725637981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: audit 2024-02-10T08:30:35.869399+0000 mon.b (mon.1) 70 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:37.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:36 smithi067 bash[24488]: audit 2024-02-10T08:30:35.869514+0000 mon.a (mon.0) 1450 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:37] "GET /metrics HTTP/1.1" 200 193187 "" "Prometheus/2.33.4" 2024-02-10T08:30:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: cluster 2024-02-10T08:30:35.973578+0000 mgr.y (mgr.14547) 335 : cluster [DBG] pgmap v364: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: audit 2024-02-10T08:30:36.860780+0000 mon.c (mon.2) 439 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: audit 2024-02-10T08:30:36.866958+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? 172.21.15.67:0/3725637981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: audit 2024-02-10T08:30:36.867235+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: cluster 2024-02-10T08:30:36.867564+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: audit 2024-02-10T08:30:36.870430+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[17385]: audit 2024-02-10T08:30:36.877372+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: cluster 2024-02-10T08:30:35.973578+0000 mgr.y (mgr.14547) 335 : cluster [DBG] pgmap v364: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: audit 2024-02-10T08:30:36.860780+0000 mon.c (mon.2) 439 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: audit 2024-02-10T08:30:36.866958+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? 172.21.15.67:0/3725637981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: audit 2024-02-10T08:30:36.867235+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: cluster 2024-02-10T08:30:36.867564+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: audit 2024-02-10T08:30:36.870430+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:37 smithi067 bash[24488]: audit 2024-02-10T08:30:36.877372+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: cluster 2024-02-10T08:30:35.973578+0000 mgr.y (mgr.14547) 335 : cluster [DBG] pgmap v364: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: audit 2024-02-10T08:30:36.860780+0000 mon.c (mon.2) 439 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: audit 2024-02-10T08:30:36.866958+0000 mon.a (mon.0) 1451 : audit [INF] from='client.? 172.21.15.67:0/3725637981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: audit 2024-02-10T08:30:36.867235+0000 mon.a (mon.0) 1452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: cluster 2024-02-10T08:30:36.867564+0000 mon.a (mon.0) 1453 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: audit 2024-02-10T08:30:36.870430+0000 mon.b (mon.1) 71 : audit [INF] from='client.? 172.21.15.67:0/3789624468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:37 smithi134 bash[18722]: audit 2024-02-10T08:30:36.877372+0000 mon.a (mon.0) 1454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]: dispatch 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.861904+0000 mon.c (mon.2) 440 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.869316+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: cluster 2024-02-10T08:30:37.869498+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.873985+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.67:0/1263582674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.874151+0000 mon.a (mon.0) 1457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.874944+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.875536+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.896755+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.897140+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.897919+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.898223+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.898719+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: audit 2024-02-10T08:30:37.899014+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[17385]: cluster 2024-02-10T08:30:37.974474+0000 mgr.y (mgr.14547) 336 : cluster [DBG] pgmap v367: 324 pgs: 64 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.861904+0000 mon.c (mon.2) 440 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.869316+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: cluster 2024-02-10T08:30:37.869498+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.873985+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.67:0/1263582674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.874151+0000 mon.a (mon.0) 1457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.874944+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.875536+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.896755+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.897140+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.897919+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.898223+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.898719+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: audit 2024-02-10T08:30:37.899014+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:39.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:38 smithi067 bash[24488]: cluster 2024-02-10T08:30:37.974474+0000 mgr.y (mgr.14547) 336 : cluster [DBG] pgmap v367: 324 pgs: 64 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.861904+0000 mon.c (mon.2) 440 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.869316+0000 mon.a (mon.0) 1455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi067-54237-32"}]': finished 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: cluster 2024-02-10T08:30:37.869498+0000 mon.a (mon.0) 1456 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.873985+0000 mon.b (mon.1) 72 : audit [INF] from='client.? 172.21.15.67:0/1263582674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.874151+0000 mon.a (mon.0) 1457 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.874944+0000 mon.c (mon.2) 441 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.875536+0000 mon.a (mon.0) 1458 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.896755+0000 mon.c (mon.2) 442 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.897140+0000 mon.a (mon.0) 1459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.897919+0000 mon.c (mon.2) 443 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.898223+0000 mon.a (mon.0) 1460 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.898719+0000 mon.c (mon.2) 444 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: audit 2024-02-10T08:30:37.899014+0000 mon.a (mon.0) 1461 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:39.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:38 smithi134 bash[18722]: cluster 2024-02-10T08:30:37.974474+0000 mgr.y (mgr.14547) 336 : cluster [DBG] pgmap v367: 324 pgs: 64 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:40.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: audit 2024-02-10T08:30:38.863160+0000 mon.c (mon.2) 445 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:40.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: audit 2024-02-10T08:30:38.881112+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: audit 2024-02-10T08:30:38.881269+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: audit 2024-02-10T08:30:38.881587+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: cluster 2024-02-10T08:30:38.881709+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: audit 2024-02-10T08:30:38.885436+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[17385]: audit 2024-02-10T08:30:38.890263+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: audit 2024-02-10T08:30:38.863160+0000 mon.c (mon.2) 445 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: audit 2024-02-10T08:30:38.881112+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: audit 2024-02-10T08:30:38.881269+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: audit 2024-02-10T08:30:38.881587+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:40.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: cluster 2024-02-10T08:30:38.881709+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-10T08:30:40.125 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: audit 2024-02-10T08:30:38.885436+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:40.125 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:39 smithi067 bash[24488]: audit 2024-02-10T08:30:38.890263+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: audit 2024-02-10T08:30:38.863160+0000 mon.c (mon.2) 445 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: audit 2024-02-10T08:30:38.881112+0000 mon.a (mon.0) 1462 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: audit 2024-02-10T08:30:38.881269+0000 mon.a (mon.0) 1463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: audit 2024-02-10T08:30:38.881587+0000 mon.a (mon.0) 1464 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi067-54237-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: cluster 2024-02-10T08:30:38.881709+0000 mon.a (mon.0) 1465 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: audit 2024-02-10T08:30:38.885436+0000 mon.c (mon.2) 446 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:39 smithi134 bash[18722]: audit 2024-02-10T08:30:38.890263+0000 mon.a (mon.0) 1466 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:39.864819+0000 mon.c (mon.2) 447 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: cluster 2024-02-10T08:30:39.975437+0000 mgr.y (mgr.14547) 337 : cluster [DBG] pgmap v370: 356 pgs: 96 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: cluster 2024-02-10T08:30:40.335985+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-10T08:30:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.337454+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.67:0/1172222526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.342244+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.348102+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.348467+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.427723+0000 mon.c (mon.2) 450 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.730229+0000 mon.c (mon.2) 451 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.735856+0000 mon.c (mon.2) 452 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[17385]: audit 2024-02-10T08:30:40.742911+0000 mon.a (mon.0) 1470 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:39.864819+0000 mon.c (mon.2) 447 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: cluster 2024-02-10T08:30:39.975437+0000 mgr.y (mgr.14547) 337 : cluster [DBG] pgmap v370: 356 pgs: 96 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: cluster 2024-02-10T08:30:40.335985+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.337454+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.67:0/1172222526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.342244+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:41.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.348102+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:41.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.348467+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:41.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.427723+0000 mon.c (mon.2) 450 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:30:41.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.730229+0000 mon.c (mon.2) 451 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:30:41.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.735856+0000 mon.c (mon.2) 452 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:30:41.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:40 smithi067 bash[24488]: audit 2024-02-10T08:30:40.742911+0000 mon.a (mon.0) 1470 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:39.864819+0000 mon.c (mon.2) 447 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: cluster 2024-02-10T08:30:39.975437+0000 mgr.y (mgr.14547) 337 : cluster [DBG] pgmap v370: 356 pgs: 96 unknown, 32 creating+peering, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: cluster 2024-02-10T08:30:40.335985+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.337454+0000 mon.c (mon.2) 448 : audit [INF] from='client.? 172.21.15.67:0/1172222526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.342244+0000 mon.a (mon.0) 1468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.348102+0000 mon.c (mon.2) 449 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.348467+0000 mon.a (mon.0) 1469 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.427723+0000 mon.c (mon.2) 450 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.730229+0000 mon.c (mon.2) 451 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.735856+0000 mon.c (mon.2) 452 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:30:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:40 smithi134 bash[18722]: audit 2024-02-10T08:30:40.742911+0000 mon.a (mon.0) 1470 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:42.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: audit 2024-02-10T08:30:40.866820+0000 mon.c (mon.2) 453 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:42.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: audit 2024-02-10T08:30:40.894066+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: audit 2024-02-10T08:30:40.894382+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: audit 2024-02-10T08:30:40.894513+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: cluster 2024-02-10T08:30:40.894614+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: audit 2024-02-10T08:30:40.897285+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[24488]: audit 2024-02-10T08:30:40.901576+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: audit 2024-02-10T08:30:40.866820+0000 mon.c (mon.2) 453 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: audit 2024-02-10T08:30:40.894066+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: audit 2024-02-10T08:30:40.894382+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: audit 2024-02-10T08:30:40.894513+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: cluster 2024-02-10T08:30:40.894614+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: audit 2024-02-10T08:30:40.897285+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:41 smithi067 bash[17385]: audit 2024-02-10T08:30:40.901576+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:42.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: audit 2024-02-10T08:30:40.866820+0000 mon.c (mon.2) 453 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: audit 2024-02-10T08:30:40.894066+0000 mon.a (mon.0) 1471 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi067-54237-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: audit 2024-02-10T08:30:40.894382+0000 mon.a (mon.0) 1472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: audit 2024-02-10T08:30:40.894513+0000 mon.a (mon.0) 1473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: cluster 2024-02-10T08:30:40.894614+0000 mon.a (mon.0) 1474 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: audit 2024-02-10T08:30:40.897285+0000 mon.c (mon.2) 454 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:41 smithi134 bash[18722]: audit 2024-02-10T08:30:40.901576+0000 mon.a (mon.0) 1475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.868545+0000 mon.c (mon.2) 455 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.892997+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]': finished 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: cluster 2024-02-10T08:30:41.893250+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.897552+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]: dispatch 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.898172+0000 mon.c (mon.2) 457 : audit [INF] from='client.? 172.21.15.67:0/1785721080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.898225+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]: dispatch 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.898700+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: audit 2024-02-10T08:30:41.967301+0000 mgr.y (mgr.14547) 338 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[17385]: cluster 2024-02-10T08:30:41.976386+0000 mgr.y (mgr.14547) 339 : cluster [DBG] pgmap v373: 396 pgs: 32 unknown, 8 creating+peering, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.868545+0000 mon.c (mon.2) 455 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.892997+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]': finished 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: cluster 2024-02-10T08:30:41.893250+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.897552+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]: dispatch 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.898172+0000 mon.c (mon.2) 457 : audit [INF] from='client.? 172.21.15.67:0/1785721080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.898225+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]: dispatch 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.898700+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: audit 2024-02-10T08:30:41.967301+0000 mgr.y (mgr.14547) 338 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:42 smithi067 bash[24488]: cluster 2024-02-10T08:30:41.976386+0000 mgr.y (mgr.14547) 339 : cluster [DBG] pgmap v373: 396 pgs: 32 unknown, 8 creating+peering, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:30:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.868545+0000 mon.c (mon.2) 455 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.892997+0000 mon.a (mon.0) 1476 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-23"}]': finished 2024-02-10T08:30:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: cluster 2024-02-10T08:30:41.893250+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-10T08:30:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.897552+0000 mon.c (mon.2) 456 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]: dispatch 2024-02-10T08:30:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.898172+0000 mon.c (mon.2) 457 : audit [INF] from='client.? 172.21.15.67:0/1785721080' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.898225+0000 mon.a (mon.0) 1478 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]: dispatch 2024-02-10T08:30:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.898700+0000 mon.a (mon.0) 1479 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: audit 2024-02-10T08:30:41.967301+0000 mgr.y (mgr.14547) 338 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:42 smithi134 bash[18722]: cluster 2024-02-10T08:30:41.976386+0000 mgr.y (mgr.14547) 339 : cluster [DBG] pgmap v373: 396 pgs: 32 unknown, 8 creating+peering, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:30:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:30:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: audit 2024-02-10T08:30:42.870161+0000 mon.c (mon.2) 458 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: cluster 2024-02-10T08:30:42.890672+0000 mon.a (mon.0) 1480 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: audit 2024-02-10T08:30:42.905356+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]': finished 2024-02-10T08:30:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: audit 2024-02-10T08:30:42.905601+0000 mon.a (mon.0) 1482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: cluster 2024-02-10T08:30:42.906229+0000 mon.a (mon.0) 1483 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: audit 2024-02-10T08:30:42.915021+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[17385]: audit 2024-02-10T08:30:42.916875+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: audit 2024-02-10T08:30:42.870161+0000 mon.c (mon.2) 458 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: cluster 2024-02-10T08:30:42.890672+0000 mon.a (mon.0) 1480 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: audit 2024-02-10T08:30:42.905356+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]': finished 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: audit 2024-02-10T08:30:42.905601+0000 mon.a (mon.0) 1482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: cluster 2024-02-10T08:30:42.906229+0000 mon.a (mon.0) 1483 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: audit 2024-02-10T08:30:42.915021+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:43 smithi067 bash[24488]: audit 2024-02-10T08:30:42.916875+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: audit 2024-02-10T08:30:42.870161+0000 mon.c (mon.2) 458 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: cluster 2024-02-10T08:30:42.890672+0000 mon.a (mon.0) 1480 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: audit 2024-02-10T08:30:42.905356+0000 mon.a (mon.0) 1481 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-23", "mode": "writeback"}]': finished 2024-02-10T08:30:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: audit 2024-02-10T08:30:42.905601+0000 mon.a (mon.0) 1482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: cluster 2024-02-10T08:30:42.906229+0000 mon.a (mon.0) 1483 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-10T08:30:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: audit 2024-02-10T08:30:42.915021+0000 mon.c (mon.2) 459 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:43 smithi134 bash[18722]: audit 2024-02-10T08:30:42.916875+0000 mon.a (mon.0) 1484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.871834+0000 mon.c (mon.2) 460 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.910506+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: cluster 2024-02-10T08:30:43.910711+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.917686+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.919271+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.67:0/1348481699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.919774+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.920653+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.950118+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: audit 2024-02-10T08:30:43.950713+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[17385]: cluster 2024-02-10T08:30:43.977413+0000 mgr.y (mgr.14547) 340 : cluster [DBG] pgmap v376: 420 pgs: 64 unknown, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.871834+0000 mon.c (mon.2) 460 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.910506+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: cluster 2024-02-10T08:30:43.910711+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.917686+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.919271+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.67:0/1348481699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:45.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.919774+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.920653+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:45.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.950118+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:45.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: audit 2024-02-10T08:30:43.950713+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:45.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:44 smithi067 bash[24488]: cluster 2024-02-10T08:30:43.977413+0000 mgr.y (mgr.14547) 340 : cluster [DBG] pgmap v376: 420 pgs: 64 unknown, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.871834+0000 mon.c (mon.2) 460 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.910506+0000 mon.a (mon.0) 1485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: cluster 2024-02-10T08:30:43.910711+0000 mon.a (mon.0) 1486 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.917686+0000 mon.c (mon.2) 461 : audit [INF] from='client.? 172.21.15.67:0/772443923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.919271+0000 mon.c (mon.2) 462 : audit [INF] from='client.? 172.21.15.67:0/1348481699' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.919774+0000 mon.a (mon.0) 1487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.920653+0000 mon.a (mon.0) 1488 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.950118+0000 mon.c (mon.2) 463 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: audit 2024-02-10T08:30:43.950713+0000 mon.a (mon.0) 1489 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:44 smithi134 bash[18722]: cluster 2024-02-10T08:30:43.977413+0000 mgr.y (mgr.14547) 340 : cluster [DBG] pgmap v376: 420 pgs: 64 unknown, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:30:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.873498+0000 mon.c (mon.2) 464 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.913934+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.914074+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.914211+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: cluster 2024-02-10T08:30:44.914285+0000 mon.a (mon.0) 1493 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.918478+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.921423+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.951941+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.952316+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.953052+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.953370+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.953942+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:44.954257+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[17385]: audit 2024-02-10T08:30:45.776191+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.873498+0000 mon.c (mon.2) 464 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.913934+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.914074+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.914211+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: cluster 2024-02-10T08:30:44.914285+0000 mon.a (mon.0) 1493 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.918478+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.921423+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.951941+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.952316+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.953052+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.953370+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.953942+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:44.954257+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:46.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:45 smithi067 bash[24488]: audit 2024-02-10T08:30:45.776191+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.873498+0000 mon.c (mon.2) 464 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.913934+0000 mon.a (mon.0) 1490 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi067-54237-33"}]': finished 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.914074+0000 mon.a (mon.0) 1491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi067-54245-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.914211+0000 mon.a (mon.0) 1492 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: cluster 2024-02-10T08:30:44.914285+0000 mon.a (mon.0) 1493 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.918478+0000 mon.c (mon.2) 465 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.921423+0000 mon.a (mon.0) 1494 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.951941+0000 mon.c (mon.2) 466 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.952316+0000 mon.a (mon.0) 1495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.953052+0000 mon.c (mon.2) 467 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.953370+0000 mon.a (mon.0) 1496 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.953942+0000 mon.c (mon.2) 468 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:44.954257+0000 mon.a (mon.0) 1497 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:45 smithi134 bash[18722]: audit 2024-02-10T08:30:45.776191+0000 mon.a (mon.0) 1498 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:30:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: audit 2024-02-10T08:30:45.875256+0000 mon.c (mon.2) 469 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: cluster 2024-02-10T08:30:45.909604+0000 mon.a (mon.0) 1499 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: audit 2024-02-10T08:30:45.922232+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]': finished 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: audit 2024-02-10T08:30:45.922456+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: cluster 2024-02-10T08:30:45.922594+0000 mon.a (mon.0) 1502 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: audit 2024-02-10T08:30:45.926506+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: audit 2024-02-10T08:30:45.929271+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[17385]: cluster 2024-02-10T08:30:45.978350+0000 mgr.y (mgr.14547) 341 : cluster [DBG] pgmap v379: 388 pgs: 32 unknown, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: audit 2024-02-10T08:30:45.875256+0000 mon.c (mon.2) 469 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: cluster 2024-02-10T08:30:45.909604+0000 mon.a (mon.0) 1499 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: audit 2024-02-10T08:30:45.922232+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]': finished 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: audit 2024-02-10T08:30:45.922456+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: cluster 2024-02-10T08:30:45.922594+0000 mon.a (mon.0) 1502 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-10T08:30:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: audit 2024-02-10T08:30:45.926506+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: audit 2024-02-10T08:30:45.929271+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:46 smithi067 bash[24488]: cluster 2024-02-10T08:30:45.978350+0000 mgr.y (mgr.14547) 341 : cluster [DBG] pgmap v379: 388 pgs: 32 unknown, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: audit 2024-02-10T08:30:45.875256+0000 mon.c (mon.2) 469 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: cluster 2024-02-10T08:30:45.909604+0000 mon.a (mon.0) 1499 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: audit 2024-02-10T08:30:45.922232+0000 mon.a (mon.0) 1500 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-23"}]': finished 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: audit 2024-02-10T08:30:45.922456+0000 mon.a (mon.0) 1501 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi067-54237-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: cluster 2024-02-10T08:30:45.922594+0000 mon.a (mon.0) 1502 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: audit 2024-02-10T08:30:45.926506+0000 mon.c (mon.2) 470 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: audit 2024-02-10T08:30:45.929271+0000 mon.a (mon.0) 1503 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:46 smithi134 bash[18722]: cluster 2024-02-10T08:30:45.978350+0000 mgr.y (mgr.14547) 341 : cluster [DBG] pgmap v379: 388 pgs: 32 unknown, 356 active+clean; 9.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:30:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:47] "GET /metrics HTTP/1.1" 200 201172 "" "Prometheus/2.33.4" 2024-02-10T08:30:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:47 smithi067 bash[17385]: audit 2024-02-10T08:30:46.876945+0000 mon.c (mon.2) 471 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:47 smithi067 bash[17385]: cluster 2024-02-10T08:30:47.379102+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-10T08:30:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:47 smithi067 bash[24488]: audit 2024-02-10T08:30:46.876945+0000 mon.c (mon.2) 471 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:47 smithi067 bash[24488]: cluster 2024-02-10T08:30:47.379102+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-10T08:30:48.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:47 smithi134 bash[18722]: audit 2024-02-10T08:30:46.876945+0000 mon.c (mon.2) 471 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:47 smithi134 bash[18722]: cluster 2024-02-10T08:30:47.379102+0000 mon.a (mon.0) 1504 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-10T08:30:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[17385]: audit 2024-02-10T08:30:47.878375+0000 mon.c (mon.2) 472 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[17385]: audit 2024-02-10T08:30:47.926751+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[17385]: cluster 2024-02-10T08:30:47.927049+0000 mon.a (mon.0) 1506 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[17385]: audit 2024-02-10T08:30:47.933444+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[17385]: audit 2024-02-10T08:30:47.943480+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[17385]: cluster 2024-02-10T08:30:47.979325+0000 mgr.y (mgr.14547) 342 : cluster [DBG] pgmap v382: 332 pgs: 40 unknown, 292 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[24488]: audit 2024-02-10T08:30:47.878375+0000 mon.c (mon.2) 472 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[24488]: audit 2024-02-10T08:30:47.926751+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[24488]: cluster 2024-02-10T08:30:47.927049+0000 mon.a (mon.0) 1506 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[24488]: audit 2024-02-10T08:30:47.933444+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[24488]: audit 2024-02-10T08:30:47.943480+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:48 smithi067 bash[24488]: cluster 2024-02-10T08:30:47.979325+0000 mgr.y (mgr.14547) 342 : cluster [DBG] pgmap v382: 332 pgs: 40 unknown, 292 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:30:49.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:48 smithi134 bash[18722]: audit 2024-02-10T08:30:47.878375+0000 mon.c (mon.2) 472 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:48 smithi134 bash[18722]: audit 2024-02-10T08:30:47.926751+0000 mon.a (mon.0) 1505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi067-54237-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:48 smithi134 bash[18722]: cluster 2024-02-10T08:30:47.927049+0000 mon.a (mon.0) 1506 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-10T08:30:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:48 smithi134 bash[18722]: audit 2024-02-10T08:30:47.933444+0000 mon.c (mon.2) 473 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:48 smithi134 bash[18722]: audit 2024-02-10T08:30:47.943480+0000 mon.a (mon.0) 1507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:48 smithi134 bash[18722]: cluster 2024-02-10T08:30:47.979325+0000 mgr.y (mgr.14547) 342 : cluster [DBG] pgmap v382: 332 pgs: 40 unknown, 292 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:30:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:49 smithi067 bash[24488]: audit 2024-02-10T08:30:48.879687+0000 mon.c (mon.2) 474 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:49 smithi067 bash[24488]: audit 2024-02-10T08:30:48.931886+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:49 smithi067 bash[24488]: cluster 2024-02-10T08:30:48.932837+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-10T08:30:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:49 smithi067 bash[17385]: audit 2024-02-10T08:30:48.879687+0000 mon.c (mon.2) 474 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:49 smithi067 bash[17385]: audit 2024-02-10T08:30:48.931886+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:50.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:49 smithi067 bash[17385]: cluster 2024-02-10T08:30:48.932837+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-10T08:30:50.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:49 smithi134 bash[18722]: audit 2024-02-10T08:30:48.879687+0000 mon.c (mon.2) 474 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:50.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:49 smithi134 bash[18722]: audit 2024-02-10T08:30:48.931886+0000 mon.a (mon.0) 1508 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:50.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:49 smithi134 bash[18722]: cluster 2024-02-10T08:30:48.932837+0000 mon.a (mon.0) 1509 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-10T08:30:51.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:49.881475+0000 mon.c (mon.2) 475 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:51.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: cluster 2024-02-10T08:30:49.951410+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:49.952780+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:49.954219+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:49.964907+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:49.965692+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: cluster 2024-02-10T08:30:49.979976+0000 mgr.y (mgr.14547) 343 : cluster [DBG] pgmap v385: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.748372+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.748710+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: cluster 2024-02-10T08:30:50.748953+0000 mon.a (mon.0) 1515 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.752921+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.753472+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.757519+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.758032+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[24488]: audit 2024-02-10T08:30:50.759687+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? 172.21.15.67:0/1311627074' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi067-54245-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:49.881475+0000 mon.c (mon.2) 475 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: cluster 2024-02-10T08:30:49.951410+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:49.952780+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:49.954219+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:49.964907+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:49.965692+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: cluster 2024-02-10T08:30:49.979976+0000 mgr.y (mgr.14547) 343 : cluster [DBG] pgmap v385: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.748372+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.748710+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: cluster 2024-02-10T08:30:50.748953+0000 mon.a (mon.0) 1515 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-10T08:30:51.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.752921+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.753472+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:51.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.757519+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.758032+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:51.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:50 smithi067 bash[17385]: audit 2024-02-10T08:30:50.759687+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? 172.21.15.67:0/1311627074' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi067-54245-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:49.881475+0000 mon.c (mon.2) 475 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: cluster 2024-02-10T08:30:49.951410+0000 mon.a (mon.0) 1510 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:49.952780+0000 mon.c (mon.2) 476 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:49.954219+0000 mon.a (mon.0) 1511 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:49.964907+0000 mon.c (mon.2) 477 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:49.965692+0000 mon.a (mon.0) 1512 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: cluster 2024-02-10T08:30:49.979976+0000 mgr.y (mgr.14547) 343 : cluster [DBG] pgmap v385: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.748372+0000 mon.a (mon.0) 1513 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.748710+0000 mon.a (mon.0) 1514 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:30:51.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: cluster 2024-02-10T08:30:50.748953+0000 mon.a (mon.0) 1515 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-10T08:30:51.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.752921+0000 mon.c (mon.2) 478 : audit [INF] from='client.? 172.21.15.67:0/3329465010' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.753472+0000 mon.c (mon.2) 479 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:51.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.757519+0000 mon.a (mon.0) 1516 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]: dispatch 2024-02-10T08:30:51.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.758032+0000 mon.a (mon.0) 1517 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:51.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:50 smithi134 bash[18722]: audit 2024-02-10T08:30:50.759687+0000 mon.a (mon.0) 1518 : audit [INF] from='client.? 172.21.15.67:0/1311627074' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi067-54245-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:50.883071+0000 mon.c (mon.2) 480 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.745399+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.745556+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]': finished 2024-02-10T08:30:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.745802+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.67:0/1311627074' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi067-54245-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: cluster 2024-02-10T08:30:51.745901+0000 mon.a (mon.0) 1522 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-10T08:30:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.751714+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.754703+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.783794+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.784699+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[24488]: audit 2024-02-10T08:30:51.785163+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi067-54237-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:50.883071+0000 mon.c (mon.2) 480 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.745399+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.745556+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]': finished 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.745802+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.67:0/1311627074' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi067-54245-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: cluster 2024-02-10T08:30:51.745901+0000 mon.a (mon.0) 1522 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.751714+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.754703+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]: dispatch 2024-02-10T08:30:52.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.783794+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.784699+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:52.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:51 smithi067 bash[17385]: audit 2024-02-10T08:30:51.785163+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi067-54237-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:52.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:50.883071+0000 mon.c (mon.2) 480 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.745399+0000 mon.a (mon.0) 1519 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi067-54237-34"}]': finished 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.745556+0000 mon.a (mon.0) 1520 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-25"}]': finished 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.745802+0000 mon.a (mon.0) 1521 : audit [INF] from='client.? 172.21.15.67:0/1311627074' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi067-54245-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: cluster 2024-02-10T08:30:51.745901+0000 mon.a (mon.0) 1522 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.751714+0000 mon.c (mon.2) 481 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]: dispatch 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.754703+0000 mon.a (mon.0) 1523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]: dispatch 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.783794+0000 mon.a (mon.0) 1524 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.784699+0000 mon.a (mon.0) 1525 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:52.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:51 smithi134 bash[18722]: audit 2024-02-10T08:30:51.785163+0000 mon.a (mon.0) 1526 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi067-54237-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: audit 2024-02-10T08:30:51.884683+0000 mon.c (mon.2) 482 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: audit 2024-02-10T08:30:51.975209+0000 mgr.y (mgr.14547) 344 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: cluster 2024-02-10T08:30:51.980733+0000 mgr.y (mgr.14547) 345 : cluster [DBG] pgmap v388: 292 pgs: 32 creating+peering, 260 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: cluster 2024-02-10T08:30:52.738995+0000 mon.a (mon.0) 1527 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: audit 2024-02-10T08:30:52.748485+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]': finished 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: audit 2024-02-10T08:30:52.748846+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi067-54237-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: cluster 2024-02-10T08:30:52.749154+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-10T08:30:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:53 smithi134 bash[18722]: audit 2024-02-10T08:30:52.750381+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi067-54237-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: audit 2024-02-10T08:30:51.884683+0000 mon.c (mon.2) 482 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: audit 2024-02-10T08:30:51.975209+0000 mgr.y (mgr.14547) 344 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: cluster 2024-02-10T08:30:51.980733+0000 mgr.y (mgr.14547) 345 : cluster [DBG] pgmap v388: 292 pgs: 32 creating+peering, 260 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: cluster 2024-02-10T08:30:52.738995+0000 mon.a (mon.0) 1527 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: audit 2024-02-10T08:30:52.748485+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]': finished 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: audit 2024-02-10T08:30:52.748846+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi067-54237-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: cluster 2024-02-10T08:30:52.749154+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[17385]: audit 2024-02-10T08:30:52.750381+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi067-54237-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: audit 2024-02-10T08:30:51.884683+0000 mon.c (mon.2) 482 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: audit 2024-02-10T08:30:51.975209+0000 mgr.y (mgr.14547) 344 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: cluster 2024-02-10T08:30:51.980733+0000 mgr.y (mgr.14547) 345 : cluster [DBG] pgmap v388: 292 pgs: 32 creating+peering, 260 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: cluster 2024-02-10T08:30:52.738995+0000 mon.a (mon.0) 1527 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: audit 2024-02-10T08:30:52.748485+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-25", "mode": "writeback"}]': finished 2024-02-10T08:30:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: audit 2024-02-10T08:30:52.748846+0000 mon.a (mon.0) 1529 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi067-54237-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:53.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: cluster 2024-02-10T08:30:52.749154+0000 mon.a (mon.0) 1530 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-10T08:30:53.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:53 smithi067 bash[24488]: audit 2024-02-10T08:30:52.750381+0000 mon.a (mon.0) 1531 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi067-54237-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:52.810409+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:52.886283+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:53.242421+0000 mon.a (mon.0) 1532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:53.887893+0000 mon.c (mon.2) 485 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:54.196609+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: cluster 2024-02-10T08:30:54.196703+0000 mon.a (mon.0) 1534 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:54.197519+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:54.198938+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.67:0/3701672535' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:54.203046+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:54.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:54 smithi134 bash[18722]: audit 2024-02-10T08:30:54.203569+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:54.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:52.810409+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:52.886283+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:53.242421+0000 mon.a (mon.0) 1532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:53.887893+0000 mon.c (mon.2) 485 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:54.196609+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: cluster 2024-02-10T08:30:54.196703+0000 mon.a (mon.0) 1534 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:54.197519+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:54.198938+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.67:0/3701672535' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:54.203046+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:54.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[17385]: audit 2024-02-10T08:30:54.203569+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:52.810409+0000 mon.c (mon.2) 483 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:52.886283+0000 mon.c (mon.2) 484 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:53.242421+0000 mon.a (mon.0) 1532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:53.887893+0000 mon.c (mon.2) 485 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:54.196609+0000 mon.a (mon.0) 1533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: cluster 2024-02-10T08:30:54.196703+0000 mon.a (mon.0) 1534 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:54.197519+0000 mon.c (mon.2) 486 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:54.198938+0000 mon.c (mon.2) 487 : audit [INF] from='client.? 172.21.15.67:0/3701672535' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:54.203046+0000 mon.a (mon.0) 1535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]: dispatch 2024-02-10T08:30:54.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:54 smithi067 bash[24488]: audit 2024-02-10T08:30:54.203569+0000 mon.a (mon.0) 1536 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: cluster 2024-02-10T08:30:53.981518+0000 mgr.y (mgr.14547) 346 : cluster [DBG] pgmap v391: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: cluster 2024-02-10T08:30:54.746837+0000 mon.a (mon.0) 1537 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: audit 2024-02-10T08:30:54.759897+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi067-54237-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: audit 2024-02-10T08:30:54.760777+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]': finished 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: audit 2024-02-10T08:30:54.761264+0000 mon.a (mon.0) 1540 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: cluster 2024-02-10T08:30:54.761422+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-10T08:30:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:55 smithi134 bash[18722]: audit 2024-02-10T08:30:54.889490+0000 mon.c (mon.2) 488 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: cluster 2024-02-10T08:30:53.981518+0000 mgr.y (mgr.14547) 346 : cluster [DBG] pgmap v391: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: cluster 2024-02-10T08:30:54.746837+0000 mon.a (mon.0) 1537 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: audit 2024-02-10T08:30:54.759897+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi067-54237-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: audit 2024-02-10T08:30:54.760777+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]': finished 2024-02-10T08:30:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: audit 2024-02-10T08:30:54.761264+0000 mon.a (mon.0) 1540 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: cluster 2024-02-10T08:30:54.761422+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[17385]: audit 2024-02-10T08:30:54.889490+0000 mon.c (mon.2) 488 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: cluster 2024-02-10T08:30:53.981518+0000 mgr.y (mgr.14547) 346 : cluster [DBG] pgmap v391: 292 pgs: 32 unknown, 260 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: cluster 2024-02-10T08:30:54.746837+0000 mon.a (mon.0) 1537 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: audit 2024-02-10T08:30:54.759897+0000 mon.a (mon.0) 1538 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi067-54237-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: audit 2024-02-10T08:30:54.760777+0000 mon.a (mon.0) 1539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-25"}]': finished 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: audit 2024-02-10T08:30:54.761264+0000 mon.a (mon.0) 1540 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi067-54245-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: cluster 2024-02-10T08:30:54.761422+0000 mon.a (mon.0) 1541 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-10T08:30:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:55 smithi067 bash[24488]: audit 2024-02-10T08:30:54.889490+0000 mon.c (mon.2) 488 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: cluster 2024-02-10T08:30:55.758280+0000 mon.a (mon.0) 1542 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:55.797029+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:55.891034+0000 mon.c (mon.2) 490 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:56.231283+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:56.232638+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:56.235295+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:56.236339+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:56 smithi134 bash[18722]: audit 2024-02-10T08:30:56.236884+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: cluster 2024-02-10T08:30:55.758280+0000 mon.a (mon.0) 1542 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:55.797029+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:55.891034+0000 mon.c (mon.2) 490 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:56.231283+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:56.232638+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:56.235295+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:56.236339+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:57.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[17385]: audit 2024-02-10T08:30:56.236884+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: cluster 2024-02-10T08:30:55.758280+0000 mon.a (mon.0) 1542 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:55.797029+0000 mon.c (mon.2) 489 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:55.891034+0000 mon.c (mon.2) 490 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:56.231283+0000 mon.a (mon.0) 1543 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:56.232638+0000 mon.c (mon.2) 491 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:56.235295+0000 mon.a (mon.0) 1544 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:56.236339+0000 mon.c (mon.2) 492 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:57.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:56 smithi067 bash[24488]: audit 2024-02-10T08:30:56.236884+0000 mon.a (mon.0) 1545 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:30:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:30:57] "GET /metrics HTTP/1.1" 200 201172 "" "Prometheus/2.33.4" 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: cluster 2024-02-10T08:30:55.982297+0000 mgr.y (mgr.14547) 347 : cluster [DBG] pgmap v394: 236 pgs: 8 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.777357+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: cluster 2024-02-10T08:30:56.777540+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.779758+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.780146+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.780702+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.781740+0000 mon.a (mon.0) 1549 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.782225+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:58 smithi134 bash[18722]: audit 2024-02-10T08:30:56.892683+0000 mon.c (mon.2) 495 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:58.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: cluster 2024-02-10T08:30:55.982297+0000 mgr.y (mgr.14547) 347 : cluster [DBG] pgmap v394: 236 pgs: 8 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.777357+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: cluster 2024-02-10T08:30:56.777540+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.779758+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.780146+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.780702+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.781740+0000 mon.a (mon.0) 1549 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.782225+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[24488]: audit 2024-02-10T08:30:56.892683+0000 mon.c (mon.2) 495 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: cluster 2024-02-10T08:30:55.982297+0000 mgr.y (mgr.14547) 347 : cluster [DBG] pgmap v394: 236 pgs: 8 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:30:58.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.777357+0000 mon.a (mon.0) 1546 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: cluster 2024-02-10T08:30:56.777540+0000 mon.a (mon.0) 1547 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.779758+0000 mon.c (mon.2) 493 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.780146+0000 mon.c (mon.2) 494 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.780702+0000 mon.a (mon.0) 1548 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.781740+0000 mon.a (mon.0) 1549 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.782225+0000 mon.a (mon.0) 1550 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:30:58.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:58 smithi067 bash[17385]: audit 2024-02-10T08:30:56.892683+0000 mon.c (mon.2) 495 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:57.777859+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:57.778039+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:57.894399+0000 mon.c (mon.2) 496 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: cluster 2024-02-10T08:30:57.983127+0000 mgr.y (mgr.14547) 348 : cluster [DBG] pgmap v397: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: cluster 2024-02-10T08:30:58.236891+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.238290+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.780342+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.780482+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: cluster 2024-02-10T08:30:58.780589+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-10T08:30:59.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.798651+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:59.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.799554+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:59.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.807067+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:30:59.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.808414+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:30:59.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.808903+0000 mon.a (mon.0) 1561 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:59.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:30:59 smithi134 bash[18722]: audit 2024-02-10T08:30:58.895864+0000 mon.c (mon.2) 498 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:57.777859+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:57.778039+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:57.894399+0000 mon.c (mon.2) 496 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: cluster 2024-02-10T08:30:57.983127+0000 mgr.y (mgr.14547) 348 : cluster [DBG] pgmap v397: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: cluster 2024-02-10T08:30:58.236891+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.238290+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.780342+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.780482+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: cluster 2024-02-10T08:30:58.780589+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-10T08:30:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.798651+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.799554+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.807067+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.808414+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.808903+0000 mon.a (mon.0) 1561 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[17385]: audit 2024-02-10T08:30:58.895864+0000 mon.c (mon.2) 498 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:57.777859+0000 mon.a (mon.0) 1551 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:57.778039+0000 mon.a (mon.0) 1552 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:57.894399+0000 mon.c (mon.2) 496 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: cluster 2024-02-10T08:30:57.983127+0000 mgr.y (mgr.14547) 348 : cluster [DBG] pgmap v397: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: cluster 2024-02-10T08:30:58.236891+0000 mon.a (mon.0) 1553 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.238290+0000 mon.a (mon.0) 1554 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]: dispatch 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.780342+0000 mon.a (mon.0) 1555 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:30:59.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.780482+0000 mon.a (mon.0) 1556 : audit [INF] from='client.? 172.21.15.67:0/2134813964' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi067-54237-35"}]': finished 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: cluster 2024-02-10T08:30:58.780589+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.798651+0000 mon.c (mon.2) 497 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.799554+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.807067+0000 mon.a (mon.0) 1559 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.808414+0000 mon.a (mon.0) 1560 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.808903+0000 mon.a (mon.0) 1561 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:30:59.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:30:59 smithi067 bash[24488]: audit 2024-02-10T08:30:58.895864+0000 mon.c (mon.2) 498 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:30:59.781904+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:30:59.782056+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: cluster 2024-02-10T08:30:59.782156+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-10T08:31:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:30:59.783468+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:30:59.784635+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi067-54237-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:30:59.786473+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:30:59.897659+0000 mon.c (mon.2) 500 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[17385]: audit 2024-02-10T08:31:00.776193+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:30:59.781904+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:30:59.782056+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: cluster 2024-02-10T08:30:59.782156+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:30:59.783468+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:30:59.784635+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi067-54237-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:30:59.786473+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:30:59.897659+0000 mon.c (mon.2) 500 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:00 smithi067 bash[24488]: audit 2024-02-10T08:31:00.776193+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:01.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:30:59.781904+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:30:59.782056+0000 mon.a (mon.0) 1563 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: cluster 2024-02-10T08:30:59.782156+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:30:59.783468+0000 mon.c (mon.2) 499 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:30:59.784635+0000 mon.a (mon.0) 1565 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi067-54237-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:30:59.786473+0000 mon.a (mon.0) 1566 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:30:59.897659+0000 mon.c (mon.2) 500 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:00 smithi134 bash[18722]: audit 2024-02-10T08:31:00.776193+0000 mon.a (mon.0) 1567 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: cluster 2024-02-10T08:30:59.983940+0000 mgr.y (mgr.14547) 349 : cluster [DBG] pgmap v400: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: audit 2024-02-10T08:31:00.899303+0000 mon.c (mon.2) 501 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: audit 2024-02-10T08:31:01.243943+0000 mon.a (mon.0) 1568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]': finished 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: cluster 2024-02-10T08:31:01.244075+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: audit 2024-02-10T08:31:01.245596+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]: dispatch 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: audit 2024-02-10T08:31:01.246058+0000 mon.c (mon.2) 503 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: audit 2024-02-10T08:31:01.246319+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]: dispatch 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[17385]: audit 2024-02-10T08:31:01.246868+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: cluster 2024-02-10T08:30:59.983940+0000 mgr.y (mgr.14547) 349 : cluster [DBG] pgmap v400: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: audit 2024-02-10T08:31:00.899303+0000 mon.c (mon.2) 501 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: audit 2024-02-10T08:31:01.243943+0000 mon.a (mon.0) 1568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]': finished 2024-02-10T08:31:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: cluster 2024-02-10T08:31:01.244075+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-10T08:31:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: audit 2024-02-10T08:31:01.245596+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]: dispatch 2024-02-10T08:31:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: audit 2024-02-10T08:31:01.246058+0000 mon.c (mon.2) 503 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: audit 2024-02-10T08:31:01.246319+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]: dispatch 2024-02-10T08:31:02.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:01 smithi067 bash[24488]: audit 2024-02-10T08:31:01.246868+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:02.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: cluster 2024-02-10T08:30:59.983940+0000 mgr.y (mgr.14547) 349 : cluster [DBG] pgmap v400: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: audit 2024-02-10T08:31:00.899303+0000 mon.c (mon.2) 501 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: audit 2024-02-10T08:31:01.243943+0000 mon.a (mon.0) 1568 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-27"}]': finished 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: cluster 2024-02-10T08:31:01.244075+0000 mon.a (mon.0) 1569 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: audit 2024-02-10T08:31:01.245596+0000 mon.c (mon.2) 502 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]: dispatch 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: audit 2024-02-10T08:31:01.246058+0000 mon.c (mon.2) 503 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: audit 2024-02-10T08:31:01.246319+0000 mon.a (mon.0) 1570 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]: dispatch 2024-02-10T08:31:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:01 smithi134 bash[18722]: audit 2024-02-10T08:31:01.246868+0000 mon.a (mon.0) 1571 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: cluster 2024-02-10T08:31:01.786024+0000 mon.a (mon.0) 1572 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.805784+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi067-54237-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.805955+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]': finished 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.806079+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: cluster 2024-02-10T08:31:01.806181+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.808211+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.816500+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.900799+0000 mon.c (mon.2) 505 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: audit 2024-02-10T08:31:01.983568+0000 mgr.y (mgr.14547) 350 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:03.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[17385]: cluster 2024-02-10T08:31:01.984765+0000 mgr.y (mgr.14547) 351 : cluster [DBG] pgmap v403: 268 pgs: 8 unknown, 260 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: cluster 2024-02-10T08:31:01.786024+0000 mon.a (mon.0) 1572 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.805784+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi067-54237-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.805955+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]': finished 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.806079+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: cluster 2024-02-10T08:31:01.806181+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.808211+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.816500+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.900799+0000 mon.c (mon.2) 505 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: audit 2024-02-10T08:31:01.983568+0000 mgr.y (mgr.14547) 350 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:03.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:02 smithi067 bash[24488]: cluster 2024-02-10T08:31:01.984765+0000 mgr.y (mgr.14547) 351 : cluster [DBG] pgmap v403: 268 pgs: 8 unknown, 260 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:31:03.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: cluster 2024-02-10T08:31:01.786024+0000 mon.a (mon.0) 1572 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.805784+0000 mon.a (mon.0) 1573 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi067-54237-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.805955+0000 mon.a (mon.0) 1574 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-27", "mode": "writeback"}]': finished 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.806079+0000 mon.a (mon.0) 1575 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: cluster 2024-02-10T08:31:01.806181+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.808211+0000 mon.c (mon.2) 504 : audit [INF] from='client.? 172.21.15.67:0/1479893921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.816500+0000 mon.a (mon.0) 1577 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]: dispatch 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.900799+0000 mon.c (mon.2) 505 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: audit 2024-02-10T08:31:01.983568+0000 mgr.y (mgr.14547) 350 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:03.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:02 smithi134 bash[18722]: cluster 2024-02-10T08:31:01.984765+0000 mgr.y (mgr.14547) 351 : cluster [DBG] pgmap v403: 268 pgs: 8 unknown, 260 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:31:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:31:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.819618+0000 mon.a (mon.0) 1578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:31:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: cluster 2024-02-10T08:31:02.819783+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.852939+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.856242+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.856752+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.874151+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.874799+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[17385]: audit 2024-02-10T08:31:02.902394+0000 mon.c (mon.2) 507 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.819618+0000 mon.a (mon.0) 1578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: cluster 2024-02-10T08:31:02.819783+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.852939+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.856242+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.856752+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:04.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.874151+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:04.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.874799+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:04.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:03 smithi067 bash[24488]: audit 2024-02-10T08:31:02.902394+0000 mon.c (mon.2) 507 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.819618+0000 mon.a (mon.0) 1578 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-45"}]': finished 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: cluster 2024-02-10T08:31:02.819783+0000 mon.a (mon.0) 1579 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.852939+0000 mon.a (mon.0) 1580 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.856242+0000 mon.a (mon.0) 1581 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.856752+0000 mon.a (mon.0) 1582 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.874151+0000 mon.c (mon.2) 506 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.874799+0000 mon.a (mon.0) 1583 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:03 smithi134 bash[18722]: audit 2024-02-10T08:31:02.902394+0000 mon.c (mon.2) 507 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.823143+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.823612+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: cluster 2024-02-10T08:31:03.823856+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.825324+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.827134+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.833874+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.835726+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.904155+0000 mon.c (mon.2) 509 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: cluster 2024-02-10T08:31:03.985676+0000 mgr.y (mgr.14547) 352 : cluster [DBG] pgmap v406: 260 pgs: 260 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.986506+0000 mon.c (mon.2) 510 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: audit 2024-02-10T08:31:03.987674+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[17385]: cluster 2024-02-10T08:31:04.816941+0000 mon.a (mon.0) 1591 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.823143+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.823612+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: cluster 2024-02-10T08:31:03.823856+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.825324+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.827134+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.833874+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.835726+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.904155+0000 mon.c (mon.2) 509 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: cluster 2024-02-10T08:31:03.985676+0000 mgr.y (mgr.14547) 352 : cluster [DBG] pgmap v406: 260 pgs: 260 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.986506+0000 mon.c (mon.2) 510 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: audit 2024-02-10T08:31:03.987674+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:05.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:05 smithi067 bash[24488]: cluster 2024-02-10T08:31:04.816941+0000 mon.a (mon.0) 1591 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.823143+0000 mon.a (mon.0) 1584 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi067-54245-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.823612+0000 mon.a (mon.0) 1585 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: cluster 2024-02-10T08:31:03.823856+0000 mon.a (mon.0) 1586 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.825324+0000 mon.c (mon.2) 508 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.827134+0000 mon.a (mon.0) 1587 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.833874+0000 mon.a (mon.0) 1588 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.835726+0000 mon.a (mon.0) 1589 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]: dispatch 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.904155+0000 mon.c (mon.2) 509 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: cluster 2024-02-10T08:31:03.985676+0000 mgr.y (mgr.14547) 352 : cluster [DBG] pgmap v406: 260 pgs: 260 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.986506+0000 mon.c (mon.2) 510 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: audit 2024-02-10T08:31:03.987674+0000 mon.a (mon.0) 1590 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:05.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:05 smithi134 bash[18722]: cluster 2024-02-10T08:31:04.816941+0000 mon.a (mon.0) 1591 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:06.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:04.827914+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:04.905548+0000 mon.c (mon.2) 511 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:05.283056+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:05.283185+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: cluster 2024-02-10T08:31:05.283288+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:05.286746+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:05.829462+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: audit 2024-02-10T08:31:05.829664+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[17385]: cluster 2024-02-10T08:31:05.830153+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:04.827914+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:04.905548+0000 mon.c (mon.2) 511 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:05.283056+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]': finished 2024-02-10T08:31:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:05.283185+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T08:31:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: cluster 2024-02-10T08:31:05.283288+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-10T08:31:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:05.286746+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:05.829462+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: audit 2024-02-10T08:31:05.829664+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:06 smithi067 bash[24488]: cluster 2024-02-10T08:31:05.830153+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-10T08:31:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:04.827914+0000 mon.a (mon.0) 1592 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:04.905548+0000 mon.c (mon.2) 511 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:05.283056+0000 mon.a (mon.0) 1593 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-27"}]': finished 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:05.283185+0000 mon.a (mon.0) 1594 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: cluster 2024-02-10T08:31:05.283288+0000 mon.a (mon.0) 1595 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:05.286746+0000 mon.a (mon.0) 1596 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]: dispatch 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:05.829462+0000 mon.a (mon.0) 1597 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi067-54245-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: audit 2024-02-10T08:31:05.829664+0000 mon.a (mon.0) 1598 : audit [INF] from='client.? 172.21.15.67:0/1653308505' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi067-54237-36"}]': finished 2024-02-10T08:31:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:06 smithi134 bash[18722]: cluster 2024-02-10T08:31:05.830153+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:05.869448+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:05.907073+0000 mon.c (mon.2) 513 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: cluster 2024-02-10T08:31:05.986671+0000 mgr.y (mgr.14547) 353 : cluster [DBG] pgmap v409: 236 pgs: 8 unknown, 228 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.015294+0000 mon.c (mon.2) 514 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.016273+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.309850+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.311037+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.320995+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.321464+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.322171+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.322533+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.844217+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]': finished 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.844501+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: cluster 2024-02-10T08:31:06.844746+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.848630+0000 mon.c (mon.2) 518 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.849189+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.852053+0000 mon.c (mon.2) 520 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.853538+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17385]: audit 2024-02-10T08:31:06.854313+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:05.869448+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:05.907073+0000 mon.c (mon.2) 513 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: cluster 2024-02-10T08:31:05.986671+0000 mgr.y (mgr.14547) 353 : cluster [DBG] pgmap v409: 236 pgs: 8 unknown, 228 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.015294+0000 mon.c (mon.2) 514 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.016273+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.309850+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.311037+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.320995+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.321464+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.322171+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.322533+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.844217+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]': finished 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.844501+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: cluster 2024-02-10T08:31:06.844746+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.848630+0000 mon.c (mon.2) 518 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.849189+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.852053+0000 mon.c (mon.2) 520 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.853538+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:07.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[24488]: audit 2024-02-10T08:31:06.854313+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:05.869448+0000 mon.c (mon.2) 512 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:05.907073+0000 mon.c (mon.2) 513 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: cluster 2024-02-10T08:31:05.986671+0000 mgr.y (mgr.14547) 353 : cluster [DBG] pgmap v409: 236 pgs: 8 unknown, 228 active+clean; 8.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.015294+0000 mon.c (mon.2) 514 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.016273+0000 mon.c (mon.2) 515 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.309850+0000 mon.a (mon.0) 1600 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.311037+0000 mon.c (mon.2) 516 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.320995+0000 mon.a (mon.0) 1601 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.321464+0000 mon.a (mon.0) 1602 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.322171+0000 mon.c (mon.2) 517 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.322533+0000 mon.a (mon.0) 1603 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.844217+0000 mon.a (mon.0) 1604 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pg_num","val":"11"}]': finished 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.844501+0000 mon.a (mon.0) 1605 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi067-54237-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: cluster 2024-02-10T08:31:06.844746+0000 mon.a (mon.0) 1606 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.848630+0000 mon.c (mon.2) 518 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.849189+0000 mon.c (mon.2) 519 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.852053+0000 mon.c (mon.2) 520 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.853538+0000 mon.a (mon.0) 1607 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:07.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:07 smithi134 bash[18722]: audit 2024-02-10T08:31:06.854313+0000 mon.a (mon.0) 1608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:31:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:07] "GET /metrics HTTP/1.1" 200 190073 "" "Prometheus/2.33.4" 2024-02-10T08:31:09.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:07.842851+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:07.847362+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:07.849976+0000 mon.c (mon.2) 522 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: cluster 2024-02-10T08:31:07.987513+0000 mgr.y (mgr.14547) 354 : cluster [DBG] pgmap v412: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:07.988208+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: cluster 2024-02-10T08:31:08.302279+0000 mon.a (mon.0) 1610 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:08.304652+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:08.306567+0000 mon.a (mon.0) 1612 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[24488]: audit 2024-02-10T08:31:08.311964+0000 mon.a (mon.0) 1613 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:07.842851+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:09.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:07.847362+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:07.849976+0000 mon.c (mon.2) 522 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: cluster 2024-02-10T08:31:07.987513+0000 mgr.y (mgr.14547) 354 : cluster [DBG] pgmap v412: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:07.988208+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: cluster 2024-02-10T08:31:08.302279+0000 mon.a (mon.0) 1610 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:08.304652+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:08.306567+0000 mon.a (mon.0) 1612 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:09.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:08 smithi067 bash[17385]: audit 2024-02-10T08:31:08.311964+0000 mon.a (mon.0) 1613 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:07.842851+0000 mon.a (mon.0) 1609 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:07.847362+0000 mon.c (mon.2) 521 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:07.849976+0000 mon.c (mon.2) 522 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: cluster 2024-02-10T08:31:07.987513+0000 mgr.y (mgr.14547) 354 : cluster [DBG] pgmap v412: 260 pgs: 32 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:07.988208+0000 mon.c (mon.2) 523 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: cluster 2024-02-10T08:31:08.302279+0000 mon.a (mon.0) 1610 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:08.304652+0000 mon.a (mon.0) 1611 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:08.306567+0000 mon.a (mon.0) 1612 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:09.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:08 smithi134 bash[18722]: audit 2024-02-10T08:31:08.311964+0000 mon.a (mon.0) 1613 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-10T08:31:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.855694+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.855889+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.856055+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.856167+0000 mon.a (mon.0) 1617 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: cluster 2024-02-10T08:31:08.856295+0000 mon.a (mon.0) 1618 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.859807+0000 mon.c (mon.2) 524 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.860743+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.866111+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[17385]: audit 2024-02-10T08:31:08.866788+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.855694+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.855889+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.856055+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.856167+0000 mon.a (mon.0) 1617 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-10T08:31:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: cluster 2024-02-10T08:31:08.856295+0000 mon.a (mon.0) 1618 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-10T08:31:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.859807+0000 mon.c (mon.2) 524 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.860743+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.866111+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:10.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:09 smithi067 bash[24488]: audit 2024-02-10T08:31:08.866788+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.855694+0000 mon.a (mon.0) 1614 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi067-54237-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.855889+0000 mon.a (mon.0) 1615 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.856055+0000 mon.a (mon.0) 1616 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.856167+0000 mon.a (mon.0) 1617 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: cluster 2024-02-10T08:31:08.856295+0000 mon.a (mon.0) 1618 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.859807+0000 mon.c (mon.2) 524 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.860743+0000 mon.c (mon.2) 525 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.866111+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]: dispatch 2024-02-10T08:31:10.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:09 smithi134 bash[18722]: audit 2024-02-10T08:31:08.866788+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:11.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.859370+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.859548+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]': finished 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: cluster 2024-02-10T08:31:09.859734+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.862822+0000 mon.c (mon.2) 526 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.863396+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]: dispatch 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.872702+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]: dispatch 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.886230+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.887117+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:09.887583+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi067-54245-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: cluster 2024-02-10T08:31:09.988447+0000 mgr.y (mgr.14547) 355 : cluster [DBG] pgmap v415: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: cluster 2024-02-10T08:31:10.743140+0000 mon.a (mon.0) 1628 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:10.755807+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]': finished 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:10.756171+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi067-54245-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:11.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: cluster 2024-02-10T08:31:10.756480+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:10.757920+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:10.762452+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi067-54245-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:10.763599+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[24488]: audit 2024-02-10T08:31:10.863985+0000 mon.c (mon.2) 529 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.859370+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.859548+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]': finished 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: cluster 2024-02-10T08:31:09.859734+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.862822+0000 mon.c (mon.2) 526 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:11.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.863396+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.872702+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.886230+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.887117+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:09.887583+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi067-54245-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: cluster 2024-02-10T08:31:09.988447+0000 mgr.y (mgr.14547) 355 : cluster [DBG] pgmap v415: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: cluster 2024-02-10T08:31:10.743140+0000 mon.a (mon.0) 1628 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:10.755807+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]': finished 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:10.756171+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi067-54245-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: cluster 2024-02-10T08:31:10.756480+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:10.757920+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:10.762452+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi067-54245-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:10.763599+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:11.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:10 smithi067 bash[17385]: audit 2024-02-10T08:31:10.863985+0000 mon.c (mon.2) 529 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.859370+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? 172.21.15.67:0/3439459366' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi067-54245-46"}]': finished 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.859548+0000 mon.a (mon.0) 1622 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-29"}]': finished 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: cluster 2024-02-10T08:31:09.859734+0000 mon.a (mon.0) 1623 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.862822+0000 mon.c (mon.2) 526 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.863396+0000 mon.c (mon.2) 527 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]: dispatch 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.872702+0000 mon.a (mon.0) 1624 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]: dispatch 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.886230+0000 mon.a (mon.0) 1625 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.887117+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:09.887583+0000 mon.a (mon.0) 1627 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi067-54245-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: cluster 2024-02-10T08:31:09.988447+0000 mgr.y (mgr.14547) 355 : cluster [DBG] pgmap v415: 268 pgs: 40 unknown, 228 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: cluster 2024-02-10T08:31:10.743140+0000 mon.a (mon.0) 1628 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:10.755807+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-29", "mode": "writeback"}]': finished 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:10.756171+0000 mon.a (mon.0) 1630 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi067-54245-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: cluster 2024-02-10T08:31:10.756480+0000 mon.a (mon.0) 1631 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:10.757920+0000 mon.c (mon.2) 528 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:10.762452+0000 mon.a (mon.0) 1632 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi067-54245-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:10.763599+0000 mon.a (mon.0) 1633 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:11.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:10 smithi134 bash[18722]: audit 2024-02-10T08:31:10.863985+0000 mon.c (mon.2) 529 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:12.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:12 smithi134 bash[18722]: audit 2024-02-10T08:31:10.895719+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:12 smithi134 bash[18722]: audit 2024-02-10T08:31:10.896349+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:12.277 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:31:11 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:31:12.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:12 smithi067 bash[17385]: audit 2024-02-10T08:31:10.895719+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:12.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:12 smithi067 bash[17385]: audit 2024-02-10T08:31:10.896349+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:12.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:12 smithi067 bash[24488]: audit 2024-02-10T08:31:10.895719+0000 mon.c (mon.2) 530 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:12.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:12 smithi067 bash[24488]: audit 2024-02-10T08:31:10.896349+0000 mon.a (mon.0) 1634 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:11.865503+0000 mon.c (mon.2) 531 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:11.988514+0000 mgr.y (mgr.14547) 356 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: cluster 2024-02-10T08:31:11.989208+0000 mgr.y (mgr.14547) 357 : cluster [DBG] pgmap v418: 259 pgs: 1 peering, 258 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.206042+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.206336+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: cluster 2024-02-10T08:31:12.206531+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.207527+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.208242+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.217229+0000 mon.a (mon.0) 1638 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.217653+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: cluster 2024-02-10T08:31:12.751069+0000 mon.a (mon.0) 1640 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.762156+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi067-54245-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.762244+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.762322+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]': finished 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: cluster 2024-02-10T08:31:12.762373+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.791437+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.792908+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.793457+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi067-54237-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:13.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:13 smithi134 bash[18722]: audit 2024-02-10T08:31:12.867427+0000 mon.c (mon.2) 534 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:11.865503+0000 mon.c (mon.2) 531 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:11.988514+0000 mgr.y (mgr.14547) 356 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: cluster 2024-02-10T08:31:11.989208+0000 mgr.y (mgr.14547) 357 : cluster [DBG] pgmap v418: 259 pgs: 1 peering, 258 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.206042+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.206336+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: cluster 2024-02-10T08:31:12.206531+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.207527+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.208242+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.217229+0000 mon.a (mon.0) 1638 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.217653+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: cluster 2024-02-10T08:31:12.751069+0000 mon.a (mon.0) 1640 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.762156+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi067-54245-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.762244+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.762322+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]': finished 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: cluster 2024-02-10T08:31:12.762373+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.791437+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:11.865503+0000 mon.c (mon.2) 531 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:11.988514+0000 mgr.y (mgr.14547) 356 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: cluster 2024-02-10T08:31:11.989208+0000 mgr.y (mgr.14547) 357 : cluster [DBG] pgmap v418: 259 pgs: 1 peering, 258 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.206042+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.206336+0000 mon.a (mon.0) 1636 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:13.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: cluster 2024-02-10T08:31:12.206531+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.207527+0000 mon.c (mon.2) 532 : audit [INF] from='client.? 172.21.15.67:0/2015806152' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.208242+0000 mon.c (mon.2) 533 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.217229+0000 mon.a (mon.0) 1638 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.217653+0000 mon.a (mon.0) 1639 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: cluster 2024-02-10T08:31:12.751069+0000 mon.a (mon.0) 1640 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.762156+0000 mon.a (mon.0) 1641 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi067-54245-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.762244+0000 mon.a (mon.0) 1642 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi067-54237-37"}]': finished 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.762322+0000 mon.a (mon.0) 1643 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-29"}]': finished 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: cluster 2024-02-10T08:31:12.762373+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.791437+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.792908+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.793457+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi067-54237-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:13.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[24488]: audit 2024-02-10T08:31:12.867427+0000 mon.c (mon.2) 534 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:13.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.792908+0000 mon.a (mon.0) 1646 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:13.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.793457+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi067-54237-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:13.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:13 smithi067 bash[17385]: audit 2024-02-10T08:31:12.867427+0000 mon.c (mon.2) 534 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:15 smithi134 bash[18722]: audit 2024-02-10T08:31:13.768667+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi067-54237-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:15 smithi134 bash[18722]: cluster 2024-02-10T08:31:13.768824+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-10T08:31:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:15 smithi134 bash[18722]: audit 2024-02-10T08:31:13.771245+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi067-54237-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:15 smithi134 bash[18722]: audit 2024-02-10T08:31:13.869222+0000 mon.c (mon.2) 535 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[17385]: audit 2024-02-10T08:31:13.768667+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi067-54237-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[17385]: cluster 2024-02-10T08:31:13.768824+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-10T08:31:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[17385]: audit 2024-02-10T08:31:13.771245+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi067-54237-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:15.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[17385]: audit 2024-02-10T08:31:13.869222+0000 mon.c (mon.2) 535 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[24488]: audit 2024-02-10T08:31:13.768667+0000 mon.a (mon.0) 1648 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi067-54237-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[24488]: cluster 2024-02-10T08:31:13.768824+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-10T08:31:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[24488]: audit 2024-02-10T08:31:13.771245+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi067-54237-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:15 smithi067 bash[24488]: audit 2024-02-10T08:31:13.869222+0000 mon.c (mon.2) 535 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: cluster 2024-02-10T08:31:13.990017+0000 mgr.y (mgr.14547) 358 : cluster [DBG] pgmap v421: 235 pgs: 8 unknown, 1 peering, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:14.870898+0000 mon.c (mon.2) 536 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: cluster 2024-02-10T08:31:15.224626+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.226492+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.227104+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.234472+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.779676+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi067-54237-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.780000+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.780201+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: cluster 2024-02-10T08:31:15.780320+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-10T08:31:16.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.781392+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:16.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.784654+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:16.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.790225+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:16.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:16 smithi134 bash[18722]: audit 2024-02-10T08:31:15.872451+0000 mon.c (mon.2) 539 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:16.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: cluster 2024-02-10T08:31:13.990017+0000 mgr.y (mgr.14547) 358 : cluster [DBG] pgmap v421: 235 pgs: 8 unknown, 1 peering, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:14.870898+0000 mon.c (mon.2) 536 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: cluster 2024-02-10T08:31:15.224626+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.226492+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.227104+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.234472+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.779676+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi067-54237-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.780000+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.780201+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: cluster 2024-02-10T08:31:15.780320+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.781392+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.784654+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.790225+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:16.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[24488]: audit 2024-02-10T08:31:15.872451+0000 mon.c (mon.2) 539 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: cluster 2024-02-10T08:31:13.990017+0000 mgr.y (mgr.14547) 358 : cluster [DBG] pgmap v421: 235 pgs: 8 unknown, 1 peering, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:14.870898+0000 mon.c (mon.2) 536 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: cluster 2024-02-10T08:31:15.224626+0000 mon.a (mon.0) 1651 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.226492+0000 mon.c (mon.2) 537 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.227104+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.234472+0000 mon.a (mon.0) 1653 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.779676+0000 mon.a (mon.0) 1654 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi067-54237-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.780000+0000 mon.a (mon.0) 1655 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.780201+0000 mon.a (mon.0) 1656 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: cluster 2024-02-10T08:31:15.780320+0000 mon.a (mon.0) 1657 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-10T08:31:16.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.781392+0000 mon.a (mon.0) 1658 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]: dispatch 2024-02-10T08:31:16.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.784654+0000 mon.c (mon.2) 538 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:16.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.790225+0000 mon.a (mon.0) 1659 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:16.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:16 smithi067 bash[17385]: audit 2024-02-10T08:31:15.872451+0000 mon.c (mon.2) 539 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:17] "GET /metrics HTTP/1.1" 200 190906 "" "Prometheus/2.33.4" 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: cluster 2024-02-10T08:31:15.990917+0000 mgr.y (mgr.14547) 359 : cluster [DBG] pgmap v424: 267 pgs: 40 unknown, 1 peering, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.785943+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.786139+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: cluster 2024-02-10T08:31:16.786301+0000 mon.a (mon.0) 1662 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.795544+0000 mon.c (mon.2) 540 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.796053+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.812495+0000 mon.c (mon.2) 541 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.813014+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.814067+0000 mon.c (mon.2) 542 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.814502+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.815429+0000 mon.c (mon.2) 543 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.815907+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[17385]: audit 2024-02-10T08:31:16.874035+0000 mon.c (mon.2) 544 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: cluster 2024-02-10T08:31:15.990917+0000 mgr.y (mgr.14547) 359 : cluster [DBG] pgmap v424: 267 pgs: 40 unknown, 1 peering, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.785943+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.786139+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: cluster 2024-02-10T08:31:16.786301+0000 mon.a (mon.0) 1662 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.795544+0000 mon.c (mon.2) 540 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.796053+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.812495+0000 mon.c (mon.2) 541 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.813014+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.814067+0000 mon.c (mon.2) 542 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.814502+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.815429+0000 mon.c (mon.2) 543 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:18.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.815907+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:18.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:17 smithi067 bash[24488]: audit 2024-02-10T08:31:16.874035+0000 mon.c (mon.2) 544 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: cluster 2024-02-10T08:31:15.990917+0000 mgr.y (mgr.14547) 359 : cluster [DBG] pgmap v424: 267 pgs: 40 unknown, 1 peering, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.785943+0000 mon.a (mon.0) 1660 : audit [INF] from='client.? 172.21.15.67:0/4291311508' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi067-54245-47"}]': finished 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.786139+0000 mon.a (mon.0) 1661 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: cluster 2024-02-10T08:31:16.786301+0000 mon.a (mon.0) 1662 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.795544+0000 mon.c (mon.2) 540 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.796053+0000 mon.a (mon.0) 1663 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.812495+0000 mon.c (mon.2) 541 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.813014+0000 mon.a (mon.0) 1664 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.814067+0000 mon.c (mon.2) 542 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.814502+0000 mon.a (mon.0) 1665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:18.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.815429+0000 mon.c (mon.2) 543 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:18.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.815907+0000 mon.a (mon.0) 1666 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:18.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:17 smithi134 bash[18722]: audit 2024-02-10T08:31:16.874035+0000 mon.c (mon.2) 544 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.789640+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]': finished 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.789867+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: cluster 2024-02-10T08:31:17.791145+0000 mon.a (mon.0) 1669 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.792473+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]: dispatch 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.794705+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.795233+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]: dispatch 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.802479+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.804946+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.875645+0000 mon.c (mon.2) 547 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.992524+0000 mon.c (mon.2) 548 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:17.994765+0000 mon.a (mon.0) 1673 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: cluster 2024-02-10T08:31:18.784406+0000 mon.a (mon.0) 1674 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:19.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:19 smithi134 bash[18722]: audit 2024-02-10T08:31:18.792767+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]': finished 2024-02-10T08:31:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.789640+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]': finished 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.789867+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: cluster 2024-02-10T08:31:17.791145+0000 mon.a (mon.0) 1669 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.792473+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.794705+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.795233+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.802479+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.804946+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.875645+0000 mon.c (mon.2) 547 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.992524+0000 mon.c (mon.2) 548 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:17.994765+0000 mon.a (mon.0) 1673 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: cluster 2024-02-10T08:31:18.784406+0000 mon.a (mon.0) 1674 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[17385]: audit 2024-02-10T08:31:18.792767+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]': finished 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.789640+0000 mon.a (mon.0) 1667 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-31"}]': finished 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.789867+0000 mon.a (mon.0) 1668 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi067-54245-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: cluster 2024-02-10T08:31:17.791145+0000 mon.a (mon.0) 1669 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.792473+0000 mon.c (mon.2) 545 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.794705+0000 mon.c (mon.2) 546 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.795233+0000 mon.a (mon.0) 1670 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.802479+0000 mon.a (mon.0) 1671 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.804946+0000 mon.a (mon.0) 1672 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.875645+0000 mon.c (mon.2) 547 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.992524+0000 mon.c (mon.2) 548 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:17.994765+0000 mon.a (mon.0) 1673 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: cluster 2024-02-10T08:31:18.784406+0000 mon.a (mon.0) 1674 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:19.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:19 smithi067 bash[24488]: audit 2024-02-10T08:31:18.792767+0000 mon.a (mon.0) 1675 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-31", "mode": "writeback"}]': finished 2024-02-10T08:31:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: cluster 2024-02-10T08:31:17.991735+0000 mgr.y (mgr.14547) 360 : cluster [DBG] pgmap v427: 259 pgs: 259 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:18.792972+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:18.877315+0000 mon.c (mon.2) 549 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.184062+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.252107+0000 mon.a (mon.0) 1677 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: cluster 2024-02-10T08:31:19.252276+0000 mon.a (mon.0) 1678 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.254795+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.255949+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.796512+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.796765+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.796963+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: cluster 2024-02-10T08:31:19.797435+0000 mon.a (mon.0) 1684 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.802101+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.805950+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.823584+0000 mon.c (mon.2) 552 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.824120+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.825108+0000 mon.c (mon.2) 553 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.825549+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.826237+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.826666+0000 mon.a (mon.0) 1688 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.878651+0000 mon.c (mon.2) 555 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.879544+0000 mon.c (mon.2) 556 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.880371+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:31:20.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:20 smithi134 bash[18722]: audit 2024-02-10T08:31:19.880794+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: cluster 2024-02-10T08:31:17.991735+0000 mgr.y (mgr.14547) 360 : cluster [DBG] pgmap v427: 259 pgs: 259 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:18.792972+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:18.877315+0000 mon.c (mon.2) 549 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.184062+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.252107+0000 mon.a (mon.0) 1677 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: cluster 2024-02-10T08:31:19.252276+0000 mon.a (mon.0) 1678 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.254795+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.255949+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.796512+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.796765+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.796963+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: cluster 2024-02-10T08:31:19.797435+0000 mon.a (mon.0) 1684 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.802101+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.805950+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.823584+0000 mon.c (mon.2) 552 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: cluster 2024-02-10T08:31:17.991735+0000 mgr.y (mgr.14547) 360 : cluster [DBG] pgmap v427: 259 pgs: 259 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:18.792972+0000 mon.a (mon.0) 1676 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:18.877315+0000 mon.c (mon.2) 549 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.184062+0000 mon.c (mon.2) 550 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:20.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.252107+0000 mon.a (mon.0) 1677 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: cluster 2024-02-10T08:31:19.252276+0000 mon.a (mon.0) 1678 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.254795+0000 mon.a (mon.0) 1679 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]: dispatch 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.255949+0000 mon.a (mon.0) 1680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.796512+0000 mon.a (mon.0) 1681 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi067-54245-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.796765+0000 mon.a (mon.0) 1682 : audit [INF] from='client.? 172.21.15.67:0/2801101463' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi067-54237-38"}]': finished 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.796963+0000 mon.a (mon.0) 1683 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: cluster 2024-02-10T08:31:19.797435+0000 mon.a (mon.0) 1684 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.802101+0000 mon.c (mon.2) 551 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.805950+0000 mon.a (mon.0) 1685 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]: dispatch 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.823584+0000 mon.c (mon.2) 552 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.824120+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.825108+0000 mon.c (mon.2) 553 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.825549+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.826237+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.826666+0000 mon.a (mon.0) 1688 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.878651+0000 mon.c (mon.2) 555 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.879544+0000 mon.c (mon.2) 556 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.880371+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[17385]: audit 2024-02-10T08:31:19.880794+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.824120+0000 mon.a (mon.0) 1686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.825108+0000 mon.c (mon.2) 553 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.825549+0000 mon.a (mon.0) 1687 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:20.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.826237+0000 mon.c (mon.2) 554 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:20.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.826666+0000 mon.a (mon.0) 1688 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:20.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.878651+0000 mon.c (mon.2) 555 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.879544+0000 mon.c (mon.2) 556 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:20.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.880371+0000 mon.c (mon.2) 557 : audit [INF] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:31:20.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:20 smithi067 bash[24488]: audit 2024-02-10T08:31:19.880794+0000 mon.a (mon.0) 1689 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-10T08:31:22.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: cluster 2024-02-10T08:31:19.992700+0000 mgr.y (mgr.14547) 361 : cluster [DBG] pgmap v430: 267 pgs: 8 unknown, 259 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: cluster 2024-02-10T08:31:20.791757+0000 mon.a (mon.0) 1690 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: audit 2024-02-10T08:31:20.806174+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]': finished 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: audit 2024-02-10T08:31:20.806398+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: audit 2024-02-10T08:31:20.806506+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: cluster 2024-02-10T08:31:20.806596+0000 mon.a (mon.0) 1694 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: audit 2024-02-10T08:31:20.812766+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: audit 2024-02-10T08:31:20.813089+0000 mon.c (mon.2) 559 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:21 smithi134 bash[18722]: audit 2024-02-10T08:31:20.813396+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: cluster 2024-02-10T08:31:19.992700+0000 mgr.y (mgr.14547) 361 : cluster [DBG] pgmap v430: 267 pgs: 8 unknown, 259 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: cluster 2024-02-10T08:31:20.791757+0000 mon.a (mon.0) 1690 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: audit 2024-02-10T08:31:20.806174+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]': finished 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: audit 2024-02-10T08:31:20.806398+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: audit 2024-02-10T08:31:20.806506+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: cluster 2024-02-10T08:31:20.806596+0000 mon.a (mon.0) 1694 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: audit 2024-02-10T08:31:20.812766+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: audit 2024-02-10T08:31:20.813089+0000 mon.c (mon.2) 559 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[24488]: audit 2024-02-10T08:31:20.813396+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: cluster 2024-02-10T08:31:19.992700+0000 mgr.y (mgr.14547) 361 : cluster [DBG] pgmap v430: 267 pgs: 8 unknown, 259 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: cluster 2024-02-10T08:31:20.791757+0000 mon.a (mon.0) 1690 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: audit 2024-02-10T08:31:20.806174+0000 mon.a (mon.0) 1691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-31"}]': finished 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: audit 2024-02-10T08:31:20.806398+0000 mon.a (mon.0) 1692 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi067-54237-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: audit 2024-02-10T08:31:20.806506+0000 mon.a (mon.0) 1693 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi067-54277-1","var":"pgp_num","val":"11"}]': finished 2024-02-10T08:31:22.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: cluster 2024-02-10T08:31:20.806596+0000 mon.a (mon.0) 1694 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-10T08:31:22.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: audit 2024-02-10T08:31:20.812766+0000 mon.c (mon.2) 558 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:22.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: audit 2024-02-10T08:31:20.813089+0000 mon.c (mon.2) 559 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:22.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:21 smithi067 bash[17385]: audit 2024-02-10T08:31:20.813396+0000 mon.a (mon.0) 1695 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: audit 2024-02-10T08:31:21.814359+0000 mon.c (mon.2) 560 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: cluster 2024-02-10T08:31:21.993574+0000 mgr.y (mgr.14547) 362 : cluster [DBG] pgmap v433: 227 pgs: 227 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: audit 2024-02-10T08:31:21.994235+0000 mon.c (mon.2) 561 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: audit 2024-02-10T08:31:21.999854+0000 mgr.y (mgr.14547) 363 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: cluster 2024-02-10T08:31:22.263377+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: audit 2024-02-10T08:31:22.264904+0000 mon.c (mon.2) 562 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: audit 2024-02-10T08:31:22.265494+0000 mon.a (mon.0) 1697 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[17385]: audit 2024-02-10T08:31:22.265826+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: audit 2024-02-10T08:31:21.814359+0000 mon.c (mon.2) 560 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: cluster 2024-02-10T08:31:21.993574+0000 mgr.y (mgr.14547) 362 : cluster [DBG] pgmap v433: 227 pgs: 227 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:23.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: audit 2024-02-10T08:31:21.994235+0000 mon.c (mon.2) 561 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:23.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: audit 2024-02-10T08:31:21.999854+0000 mgr.y (mgr.14547) 363 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:23.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: cluster 2024-02-10T08:31:22.263377+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-10T08:31:23.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: audit 2024-02-10T08:31:22.264904+0000 mon.c (mon.2) 562 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:23.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: audit 2024-02-10T08:31:22.265494+0000 mon.a (mon.0) 1697 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:23.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:22 smithi067 bash[24488]: audit 2024-02-10T08:31:22.265826+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: audit 2024-02-10T08:31:21.814359+0000 mon.c (mon.2) 560 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: cluster 2024-02-10T08:31:21.993574+0000 mgr.y (mgr.14547) 362 : cluster [DBG] pgmap v433: 227 pgs: 227 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: audit 2024-02-10T08:31:21.994235+0000 mon.c (mon.2) 561 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: audit 2024-02-10T08:31:21.999854+0000 mgr.y (mgr.14547) 363 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: cluster 2024-02-10T08:31:22.263377+0000 mon.a (mon.0) 1696 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: audit 2024-02-10T08:31:22.264904+0000 mon.c (mon.2) 562 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: audit 2024-02-10T08:31:22.265494+0000 mon.a (mon.0) 1697 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-02-10T08:31:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:22 smithi134 bash[18722]: audit 2024-02-10T08:31:22.265826+0000 mon.a (mon.0) 1698 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:31:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.815767+0000 mon.c (mon.2) 563 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.824347+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.824544+0000 mon.a (mon.0) 1700 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.824671+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: cluster 2024-02-10T08:31:22.824803+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.826269+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.826637+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.829818+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[24488]: audit 2024-02-10T08:31:22.830838+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.815767+0000 mon.c (mon.2) 563 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.824347+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.824544+0000 mon.a (mon.0) 1700 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-10T08:31:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.824671+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: cluster 2024-02-10T08:31:22.824803+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-10T08:31:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.826269+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.826637+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.829818+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:24.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:23 smithi067 bash[17385]: audit 2024-02-10T08:31:22.830838+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.815767+0000 mon.c (mon.2) 563 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.824347+0000 mon.a (mon.0) 1699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi067-54237-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.824544+0000 mon.a (mon.0) 1700 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi067-54277-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.824671+0000 mon.a (mon.0) 1701 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: cluster 2024-02-10T08:31:22.824803+0000 mon.a (mon.0) 1702 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.826269+0000 mon.c (mon.2) 564 : audit [INF] from='client.? 172.21.15.67:0/3368764901' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.826637+0000 mon.c (mon.2) 565 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.829818+0000 mon.a (mon.0) 1703 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]: dispatch 2024-02-10T08:31:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:23 smithi134 bash[18722]: audit 2024-02-10T08:31:22.830838+0000 mon.a (mon.0) 1704 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.816985+0000 mon.c (mon.2) 566 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.824666+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.824820+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: cluster 2024-02-10T08:31:23.825017+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.834015+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.839088+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.865521+0000 mon.c (mon.2) 568 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.865949+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.866530+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.866868+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.867475+0000 mon.c (mon.2) 570 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: audit 2024-02-10T08:31:23.867870+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[24488]: cluster 2024-02-10T08:31:23.994632+0000 mgr.y (mgr.14547) 364 : cluster [DBG] pgmap v436: 267 pgs: 40 unknown, 227 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.816985+0000 mon.c (mon.2) 566 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.824666+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.824820+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: cluster 2024-02-10T08:31:23.825017+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.834015+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.839088+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.865521+0000 mon.c (mon.2) 568 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.865949+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.866530+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.866868+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.867475+0000 mon.c (mon.2) 570 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: audit 2024-02-10T08:31:23.867870+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:25.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:24 smithi067 bash[17385]: cluster 2024-02-10T08:31:23.994632+0000 mgr.y (mgr.14547) 364 : cluster [DBG] pgmap v436: 267 pgs: 40 unknown, 227 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:25.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.816985+0000 mon.c (mon.2) 566 : audit [DBG] from='client.? 172.21.15.67:0/1321425724' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.824666+0000 mon.a (mon.0) 1705 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi067-54245-48"}]': finished 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.824820+0000 mon.a (mon.0) 1706 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: cluster 2024-02-10T08:31:23.825017+0000 mon.a (mon.0) 1707 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.834015+0000 mon.c (mon.2) 567 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.839088+0000 mon.a (mon.0) 1708 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.865521+0000 mon.c (mon.2) 568 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.865949+0000 mon.a (mon.0) 1709 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.866530+0000 mon.c (mon.2) 569 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.866868+0000 mon.a (mon.0) 1710 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.867475+0000 mon.c (mon.2) 570 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: audit 2024-02-10T08:31:23.867870+0000 mon.a (mon.0) 1711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:25.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:24 smithi134 bash[18722]: cluster 2024-02-10T08:31:23.994632+0000 mgr.y (mgr.14547) 364 : cluster [DBG] pgmap v436: 267 pgs: 40 unknown, 227 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:31:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.833826+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.834053+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: cluster 2024-02-10T08:31:24.834171+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.836157+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.836485+0000 mon.c (mon.2) 572 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.838377+0000 mon.c (mon.2) 573 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.843738+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.844423+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[17385]: audit 2024-02-10T08:31:24.844981+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.833826+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:26.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.834053+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: cluster 2024-02-10T08:31:24.834171+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.836157+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.836485+0000 mon.c (mon.2) 572 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.838377+0000 mon.c (mon.2) 573 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.843738+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.844423+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:26.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:26 smithi067 bash[24488]: audit 2024-02-10T08:31:24.844981+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:26.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.833826+0000 mon.a (mon.0) 1712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.834053+0000 mon.a (mon.0) 1713 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi067-54245-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: cluster 2024-02-10T08:31:24.834171+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.836157+0000 mon.c (mon.2) 571 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.836485+0000 mon.c (mon.2) 572 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.838377+0000 mon.c (mon.2) 573 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.843738+0000 mon.a (mon.0) 1715 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.844423+0000 mon.a (mon.0) 1716 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:26 smithi134 bash[18722]: audit 2024-02-10T08:31:24.844981+0000 mon.a (mon.0) 1717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:25.832533+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:25.840270+0000 mon.c (mon.2) 574 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]: dispatch 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: cluster 2024-02-10T08:31:25.995412+0000 mgr.y (mgr.14547) 365 : cluster [DBG] pgmap v439: 258 pgs: 32 unknown, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.284274+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: cluster 2024-02-10T08:31:26.284458+0000 mon.a (mon.0) 1720 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.285475+0000 mon.c (mon.2) 575 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.286063+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]: dispatch 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.287464+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: cluster 2024-02-10T08:31:26.831241+0000 mon.a (mon.0) 1723 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:27.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.841853+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.842004+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]': finished 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.842155+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: cluster 2024-02-10T08:31:26.842254+0000 mon.a (mon.0) 1727 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.877832+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.877913+0000 mon.c (mon.2) 576 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.879420+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:25.832533+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:27.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:25.840270+0000 mon.c (mon.2) 574 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]: dispatch 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: cluster 2024-02-10T08:31:25.995412+0000 mgr.y (mgr.14547) 365 : cluster [DBG] pgmap v439: 258 pgs: 32 unknown, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.284274+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: cluster 2024-02-10T08:31:26.284458+0000 mon.a (mon.0) 1720 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.285475+0000 mon.c (mon.2) 575 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.286063+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]: dispatch 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.287464+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: cluster 2024-02-10T08:31:26.831241+0000 mon.a (mon.0) 1723 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.841853+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.842004+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]': finished 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.842155+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: cluster 2024-02-10T08:31:26.842254+0000 mon.a (mon.0) 1727 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.877832+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:27.625 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.877913+0000 mon.c (mon.2) 576 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.879420+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.879655+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.880202+0000 mon.c (mon.2) 577 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.880283+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi067-54237-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.880724+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.881464+0000 mon.c (mon.2) 578 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[24488]: audit 2024-02-10T08:31:26.881818+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.879655+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.880202+0000 mon.c (mon.2) 577 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.880283+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi067-54237-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.880724+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.626 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.881464+0000 mon.c (mon.2) 578 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.627 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17385]: audit 2024-02-10T08:31:26.881818+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:25.832533+0000 mon.a (mon.0) 1718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:25.840270+0000 mon.c (mon.2) 574 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]: dispatch 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: cluster 2024-02-10T08:31:25.995412+0000 mgr.y (mgr.14547) 365 : cluster [DBG] pgmap v439: 258 pgs: 32 unknown, 226 active+clean; 8.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.284274+0000 mon.a (mon.0) 1719 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: cluster 2024-02-10T08:31:26.284458+0000 mon.a (mon.0) 1720 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.285475+0000 mon.c (mon.2) 575 : audit [INF] from='client.? 172.21.15.67:0/3496509622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.286063+0000 mon.a (mon.0) 1721 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.287464+0000 mon.a (mon.0) 1722 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: cluster 2024-02-10T08:31:26.831241+0000 mon.a (mon.0) 1723 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.841853+0000 mon.a (mon.0) 1724 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi067-54245-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.842004+0000 mon.a (mon.0) 1725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-33", "mode": "writeback"}]': finished 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.842155+0000 mon.a (mon.0) 1726 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi067-54237-39"}]': finished 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: cluster 2024-02-10T08:31:26.842254+0000 mon.a (mon.0) 1727 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.877832+0000 mon.a (mon.0) 1728 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.877913+0000 mon.c (mon.2) 576 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.879420+0000 mon.a (mon.0) 1729 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.879655+0000 mon.a (mon.0) 1730 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.880202+0000 mon.c (mon.2) 577 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.880283+0000 mon.a (mon.0) 1731 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi067-54237-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.880724+0000 mon.a (mon.0) 1732 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:27.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.881464+0000 mon.c (mon.2) 578 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:27.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:27 smithi134 bash[18722]: audit 2024-02-10T08:31:26.881818+0000 mon.a (mon.0) 1733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:31:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:27] "GET /metrics HTTP/1.1" 200 190906 "" "Prometheus/2.33.4" 2024-02-10T08:31:29.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: audit 2024-02-10T08:31:27.849340+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi067-54237-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:29.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: audit 2024-02-10T08:31:27.849513+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:29.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: cluster 2024-02-10T08:31:27.849606+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: audit 2024-02-10T08:31:27.850974+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi067-54237-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: audit 2024-02-10T08:31:27.854423+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: audit 2024-02-10T08:31:27.855270+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[17385]: cluster 2024-02-10T08:31:27.996261+0000 mgr.y (mgr.14547) 366 : cluster [DBG] pgmap v442: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: audit 2024-02-10T08:31:27.849340+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi067-54237-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: audit 2024-02-10T08:31:27.849513+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: cluster 2024-02-10T08:31:27.849606+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: audit 2024-02-10T08:31:27.850974+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi067-54237-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: audit 2024-02-10T08:31:27.854423+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: audit 2024-02-10T08:31:27.855270+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:28 smithi067 bash[24488]: cluster 2024-02-10T08:31:27.996261+0000 mgr.y (mgr.14547) 366 : cluster [DBG] pgmap v442: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:29.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: audit 2024-02-10T08:31:27.849340+0000 mon.a (mon.0) 1734 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi067-54237-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: audit 2024-02-10T08:31:27.849513+0000 mon.a (mon.0) 1735 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi067-54277-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: cluster 2024-02-10T08:31:27.849606+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-10T08:31:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: audit 2024-02-10T08:31:27.850974+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi067-54237-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: audit 2024-02-10T08:31:27.854423+0000 mon.c (mon.2) 579 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: audit 2024-02-10T08:31:27.855270+0000 mon.a (mon.0) 1738 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:28 smithi134 bash[18722]: cluster 2024-02-10T08:31:27.996261+0000 mgr.y (mgr.14547) 366 : cluster [DBG] pgmap v442: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:29 smithi134 bash[18722]: cluster 2024-02-10T08:31:29.750056+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-10T08:31:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:29 smithi134 bash[18722]: audit 2024-02-10T08:31:29.751420+0000 mon.c (mon.2) 580 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:29 smithi134 bash[18722]: audit 2024-02-10T08:31:29.751899+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:29 smithi134 bash[18722]: audit 2024-02-10T08:31:29.813222+0000 mon.c (mon.2) 581 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:29 smithi134 bash[18722]: audit 2024-02-10T08:31:29.813731+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[17385]: cluster 2024-02-10T08:31:29.750056+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-10T08:31:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[17385]: audit 2024-02-10T08:31:29.751420+0000 mon.c (mon.2) 580 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[17385]: audit 2024-02-10T08:31:29.751899+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[17385]: audit 2024-02-10T08:31:29.813222+0000 mon.c (mon.2) 581 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[17385]: audit 2024-02-10T08:31:29.813731+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[24488]: cluster 2024-02-10T08:31:29.750056+0000 mon.a (mon.0) 1739 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[24488]: audit 2024-02-10T08:31:29.751420+0000 mon.c (mon.2) 580 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[24488]: audit 2024-02-10T08:31:29.751899+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[24488]: audit 2024-02-10T08:31:29.813222+0000 mon.c (mon.2) 581 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:29 smithi067 bash[24488]: audit 2024-02-10T08:31:29.813731+0000 mon.a (mon.0) 1741 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAintry=9 expected=9 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:de5d7c5f:::12:head 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:de5d7c5f:::12:head expected=12:de5d7c5f:::12:head 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:de5d7c5f:::12:head -> 12 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=12 expected=12 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:d83876eb:::4:head 2024-02-10T08:31:31.198 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:d83876eb:::4:head expected=12:d83876eb:::4:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:d83876eb:::4:head -> 4 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=4 expected=4 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:cfc208b3:::3:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:cfc208b3:::3:head expected=12:cfc208b3:::3:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:cfc208b3:::3:head -> 3 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=3 expected=3 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:c4fdafeb:::6:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:c4fdafeb:::6:head expected=12:c4fdafeb:::6:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:c4fdafeb:::6:head -> 6 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=6 expected=6 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:89d3ae78:::11:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:89d3ae78:::11:head expected=12:89d3ae78:::11:head 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:89d3ae78:::11:head -> 11 2024-02-10T08:31:31.199 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=11 expected=11 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:863748b0:::15:head 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:863748b0:::15:head expected=12:863748b0:::15:head 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:863748b0:::15:head -> 15 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=15 expected=15 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:6cac518f:::0:head 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:6cac518f:::0:head expected=12:6cac518f:::0:head 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:6cac518f:::0:head -> 0 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=0 expected=0 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : seek to 12:02547ec2:::1:head 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : cursor()=12:02547ec2:::1:head expected=12:02547ec2:::1:head 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: > 12:02547ec2:::1:head -> 1 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: : entry=1 expected=1 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (406 ms) 2024-02-10T08:31:31.200 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (156748 ms) 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: split 0/5 -> MIN 12:33333333::::head 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: split 1/5 -> 12:33333333::::head 12:66666666::::head 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: split 2/5 -> 12:66666666::::head 12:99999999::::head 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: split 3/5 -> 12:99999999::::head 12:cccccccc::::head 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: split 4/5 -> 12:cccccccc::::head MAX 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (20134 ms) 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] 7 tests from LibRadosList (177573 ms total) 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1271 ms) 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset foo1,foo2,foo3 2024-02-10T08:31:31.201 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo1 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo2 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo3 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset foo1,foo4,foo5 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo4 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo5 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo1 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset foo6,foo7 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo7 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: foo6 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns1:foo4 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns1:foo5 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns2:foo7 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns2:foo6 2024-02-10T08:31:31.202 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: ns1:foo1 2024-02-10T08:31:31.203 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: :foo1 2024-02-10T08:31:31.203 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: :foo2 2024-02-10T08:31:31.203 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: :foo3 2024-02-10T08:31:31.203 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (57 ms) 2024-02-10T08:31:31.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.867667+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi067-54237-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.867821+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.867952+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.868072+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: cluster 2024-02-10T08:31:29.868167+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.868960+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.872459+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.873325+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:29.879835+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: cluster 2024-02-10T08:31:29.997122+0000 mgr.y (mgr.14547) 367 : cluster [DBG] pgmap v445: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:30.779087+0000 mon.a (mon.0) 1749 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:30.867409+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: audit 2024-02-10T08:31:30.867884+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:30 smithi134 bash[18722]: cluster 2024-02-10T08:31:30.868028+0000 mon.a (mon.0) 1752 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-10T08:31:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.867667+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi067-54237-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.867821+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.867952+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.868072+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: cluster 2024-02-10T08:31:29.868167+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.868960+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.872459+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.873325+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:29.879835+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: cluster 2024-02-10T08:31:29.997122+0000 mgr.y (mgr.14547) 367 : cluster [DBG] pgmap v445: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:30.779087+0000 mon.a (mon.0) 1749 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:30.867409+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: audit 2024-02-10T08:31:30.867884+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[17385]: cluster 2024-02-10T08:31:30.868028+0000 mon.a (mon.0) 1752 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.867667+0000 mon.a (mon.0) 1742 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi067-54237-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.867821+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi067-54277-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.867952+0000 mon.a (mon.0) 1744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.868072+0000 mon.a (mon.0) 1745 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: cluster 2024-02-10T08:31:29.868167+0000 mon.a (mon.0) 1746 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.868960+0000 mon.c (mon.2) 582 : audit [INF] from='client.? 172.21.15.67:0/3410550273' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.872459+0000 mon.a (mon.0) 1747 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]: dispatch 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.873325+0000 mon.c (mon.2) 583 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:29.879835+0000 mon.a (mon.0) 1748 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:31.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: cluster 2024-02-10T08:31:29.997122+0000 mgr.y (mgr.14547) 367 : cluster [DBG] pgmap v445: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:31.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:30.779087+0000 mon.a (mon.0) 1749 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:31.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:30.867409+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi067-54245-49"}]': finished 2024-02-10T08:31:31.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: audit 2024-02-10T08:31:30.867884+0000 mon.a (mon.0) 1751 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:31.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:30 smithi067 bash[24488]: cluster 2024-02-10T08:31:30.868028+0000 mon.a (mon.0) 1752 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-10T08:31:32.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:31:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: cluster 2024-02-10T08:31:31.876921+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-10T08:31:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:31.880604+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:31.881033+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? 172.21.15.67:0/4199303164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:33.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:31.881326+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:33.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:31.882238+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:33.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:31.915122+0000 mon.c (mon.2) 585 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:33.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:31.915797+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:33.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: cluster 2024-02-10T08:31:31.998132+0000 mgr.y (mgr.14547) 368 : cluster [DBG] pgmap v448: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:31:33.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:32 smithi134 bash[18722]: audit 2024-02-10T08:31:32.007099+0000 mgr.y (mgr.14547) 369 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:33.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: cluster 2024-02-10T08:31:31.876921+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-10T08:31:33.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:31.880604+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:33.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:31.881033+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? 172.21.15.67:0/4199303164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:31.881326+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:31.882238+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:31.915122+0000 mon.c (mon.2) 585 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:31.915797+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: cluster 2024-02-10T08:31:31.998132+0000 mgr.y (mgr.14547) 368 : cluster [DBG] pgmap v448: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[24488]: audit 2024-02-10T08:31:32.007099+0000 mgr.y (mgr.14547) 369 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: cluster 2024-02-10T08:31:31.876921+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:31.880604+0000 mon.a (mon.0) 1754 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:31.881033+0000 mon.a (mon.0) 1755 : audit [INF] from='client.? 172.21.15.67:0/4199303164' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:31.881326+0000 mon.c (mon.2) 584 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:31.882238+0000 mon.a (mon.0) 1756 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:33.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:31.915122+0000 mon.c (mon.2) 585 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:33.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:31.915797+0000 mon.a (mon.0) 1757 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:33.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: cluster 2024-02-10T08:31:31.998132+0000 mgr.y (mgr.14547) 368 : cluster [DBG] pgmap v448: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:31:33.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:32 smithi067 bash[17385]: audit 2024-02-10T08:31:32.007099+0000 mgr.y (mgr.14547) 369 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.872298+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.872547+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? 172.21.15.67:0/4199303164' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.872716+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.873034+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: cluster 2024-02-10T08:31:32.873166+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.875773+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.876936+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.877158+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.877586+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:32.887607+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: cluster 2024-02-10T08:31:33.868185+0000 mon.a (mon.0) 1766 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:33.875651+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:33.875859+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: audit 2024-02-10T08:31:33.875981+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:34.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:33 smithi134 bash[18722]: cluster 2024-02-10T08:31:33.876152+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-10T08:31:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.872298+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.872547+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? 172.21.15.67:0/4199303164' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.872716+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.873034+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: cluster 2024-02-10T08:31:32.873166+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.875773+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.876936+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.877158+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.877586+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:32.887607+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:34.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: cluster 2024-02-10T08:31:33.868185+0000 mon.a (mon.0) 1766 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:33.875651+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:33.875859+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: audit 2024-02-10T08:31:33.875981+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[17385]: cluster 2024-02-10T08:31:33.876152+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.872298+0000 mon.a (mon.0) 1758 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.872547+0000 mon.a (mon.0) 1759 : audit [INF] from='client.? 172.21.15.67:0/4199303164' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi067-54245-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.872716+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.873034+0000 mon.a (mon.0) 1761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: cluster 2024-02-10T08:31:32.873166+0000 mon.a (mon.0) 1762 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-10T08:31:34.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.875773+0000 mon.c (mon.2) 586 : audit [INF] from='client.? 172.21.15.67:0/3544559772' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.876936+0000 mon.c (mon.2) 587 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.877158+0000 mon.a (mon.0) 1763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]: dispatch 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.877586+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]: dispatch 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:32.887607+0000 mon.a (mon.0) 1765 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]: dispatch 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: cluster 2024-02-10T08:31:33.868185+0000 mon.a (mon.0) 1766 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:33.875651+0000 mon.a (mon.0) 1767 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi067-54277-2"}]': finished 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:33.875859+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? 172.21.15.67:0/2297346358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi067-54237-40"}]': finished 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: audit 2024-02-10T08:31:33.875981+0000 mon.a (mon.0) 1769 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-33"}]': finished 2024-02-10T08:31:34.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:33 smithi067 bash[24488]: cluster 2024-02-10T08:31:33.876152+0000 mon.a (mon.0) 1770 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.909909+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.910110+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.913924+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.913954+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.916350+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.916509+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.916968+0000 mon.c (mon.2) 588 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.917703+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.918372+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.918685+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.919212+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: audit 2024-02-10T08:31:33.919559+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:34 smithi134 bash[18722]: cluster 2024-02-10T08:31:33.998980+0000 mgr.y (mgr.14547) 370 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:31:35.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.909909+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.910110+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.913924+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.913954+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.916350+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.916509+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.916968+0000 mon.c (mon.2) 588 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.917703+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.918372+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.918685+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.919212+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: audit 2024-02-10T08:31:33.919559+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[24488]: cluster 2024-02-10T08:31:33.998980+0000 mgr.y (mgr.14547) 370 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.909909+0000 mon.b (mon.1) 73 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.910110+0000 mon.a (mon.0) 1771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.913924+0000 mon.b (mon.1) 74 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.913954+0000 mon.a (mon.0) 1772 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.916350+0000 mon.b (mon.1) 75 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.916509+0000 mon.a (mon.0) 1773 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.916968+0000 mon.c (mon.2) 588 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.917703+0000 mon.a (mon.0) 1774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.918372+0000 mon.c (mon.2) 589 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.918685+0000 mon.a (mon.0) 1775 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:35.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.919212+0000 mon.c (mon.2) 590 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: audit 2024-02-10T08:31:33.919559+0000 mon.a (mon.0) 1776 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:35.373 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:34 smithi067 bash[17385]: cluster 2024-02-10T08:31:33.998980+0000 mgr.y (mgr.14547) 370 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:31:36.939 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-10T08:31:36.939 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 1 0 2024-02-10T08:31:36.939 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 10 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 13 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 7 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 14 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 0 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 15 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 11 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 5 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 8 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 6 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 3 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 4 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 12 0 2024-02-10T08:31:36.940 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 9 0 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 2 0 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (77 ms) 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] 3 tests from LibRadosListEC (1405 ms total) 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3056 ms) 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] 1 test from LibRadosListNP (3056 ms total) 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [----------] Global test environment tear-down 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [==========] 11 tests from 3 test suites ran. (200142 ms total) 2024-02-10T08:31:36.941 INFO:tasks.workunit.client.0.smithi067.stdout: api_list: [ PASSED ] 11 tests. 2024-02-10T08:31:37.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.915146+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.915262+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: cluster 2024-02-10T08:31:34.915415+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.918565+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.920720+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.921277+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54277-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.925744+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[24488]: audit 2024-02-10T08:31:34.926856+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.915146+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.915262+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: cluster 2024-02-10T08:31:34.915415+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-10T08:31:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.918565+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.920720+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.921277+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54277-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.925744+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:36 smithi067 bash[17385]: audit 2024-02-10T08:31:34.926856+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:37.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.915146+0000 mon.a (mon.0) 1777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.915262+0000 mon.a (mon.0) 1778 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi067-54237-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: cluster 2024-02-10T08:31:34.915415+0000 mon.a (mon.0) 1779 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.918565+0000 mon.c (mon.2) 591 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.920720+0000 mon.b (mon.1) 76 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.921277+0000 mon.a (mon.0) 1780 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54277-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.925744+0000 mon.a (mon.0) 1781 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:36 smithi134 bash[18722]: audit 2024-02-10T08:31:34.926856+0000 mon.a (mon.0) 1782 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:37] "GET /metrics HTTP/1.1" 200 192184 "" "Prometheus/2.33.4" 2024-02-10T08:31:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:35.921471+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54277-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: cluster 2024-02-10T08:31:35.999755+0000 mgr.y (mgr.14547) 371 : cluster [DBG] pgmap v454: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: cluster 2024-02-10T08:31:36.818092+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-10T08:31:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.819613+0000 mon.c (mon.2) 592 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.829720+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi067-54277-3","pool2":"test-rados-api-smithi067-54277-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.830374+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.924756+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.924877+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.925088+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi067-54277-3","pool2":"test-rados-api-smithi067-54277-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.925185+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: cluster 2024-02-10T08:31:36.925265+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.931967+0000 mon.c (mon.2) 593 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[24488]: audit 2024-02-10T08:31:36.935067+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:35.921471+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54277-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: cluster 2024-02-10T08:31:35.999755+0000 mgr.y (mgr.14547) 371 : cluster [DBG] pgmap v454: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: cluster 2024-02-10T08:31:36.818092+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.819613+0000 mon.c (mon.2) 592 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.829720+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi067-54277-3","pool2":"test-rados-api-smithi067-54277-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.830374+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.924756+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.924877+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.925088+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi067-54277-3","pool2":"test-rados-api-smithi067-54277-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.925185+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: cluster 2024-02-10T08:31:36.925265+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.931967+0000 mon.c (mon.2) 593 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:38.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:37 smithi067 bash[17385]: audit 2024-02-10T08:31:36.935067+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:35.921471+0000 mon.a (mon.0) 1783 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54277-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: cluster 2024-02-10T08:31:35.999755+0000 mgr.y (mgr.14547) 371 : cluster [DBG] pgmap v454: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: cluster 2024-02-10T08:31:36.818092+0000 mon.a (mon.0) 1784 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.819613+0000 mon.c (mon.2) 592 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.829720+0000 mon.a (mon.0) 1785 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi067-54277-3","pool2":"test-rados-api-smithi067-54277-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.830374+0000 mon.a (mon.0) 1786 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.924756+0000 mon.a (mon.0) 1787 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi067-54237-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.924877+0000 mon.a (mon.0) 1788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi067-54245-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.925088+0000 mon.a (mon.0) 1789 : audit [INF] from='client.? 172.21.15.67:0/139703393' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi067-54277-3","pool2":"test-rados-api-smithi067-54277-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.925185+0000 mon.a (mon.0) 1790 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: cluster 2024-02-10T08:31:36.925265+0000 mon.a (mon.0) 1791 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-10T08:31:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.931967+0000 mon.c (mon.2) 593 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:38.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:37 smithi134 bash[18722]: audit 2024-02-10T08:31:36.935067+0000 mon.a (mon.0) 1792 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:39.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:38 smithi134 bash[18722]: audit 2024-02-10T08:31:37.928502+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:38 smithi134 bash[18722]: cluster 2024-02-10T08:31:37.928783+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-10T08:31:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:38 smithi134 bash[18722]: audit 2024-02-10T08:31:37.934136+0000 mon.c (mon.2) 594 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:38 smithi134 bash[18722]: audit 2024-02-10T08:31:37.934736+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:38 smithi134 bash[18722]: cluster 2024-02-10T08:31:38.000678+0000 mgr.y (mgr.14547) 372 : cluster [DBG] pgmap v457: 244 pgs: 25 creating+peering, 23 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T08:31:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[24488]: audit 2024-02-10T08:31:37.928502+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[24488]: cluster 2024-02-10T08:31:37.928783+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-10T08:31:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[24488]: audit 2024-02-10T08:31:37.934136+0000 mon.c (mon.2) 594 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[24488]: audit 2024-02-10T08:31:37.934736+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[24488]: cluster 2024-02-10T08:31:38.000678+0000 mgr.y (mgr.14547) 372 : cluster [DBG] pgmap v457: 244 pgs: 25 creating+peering, 23 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T08:31:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[17385]: audit 2024-02-10T08:31:37.928502+0000 mon.a (mon.0) 1793 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[17385]: cluster 2024-02-10T08:31:37.928783+0000 mon.a (mon.0) 1794 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-10T08:31:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[17385]: audit 2024-02-10T08:31:37.934136+0000 mon.c (mon.2) 594 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[17385]: audit 2024-02-10T08:31:37.934736+0000 mon.a (mon.0) 1795 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:38 smithi067 bash[17385]: cluster 2024-02-10T08:31:38.000678+0000 mgr.y (mgr.14547) 372 : cluster [DBG] pgmap v457: 244 pgs: 25 creating+peering, 23 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.938192+0000 mon.a (mon.0) 1796 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]': finished 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: cluster 2024-02-10T08:31:38.938341+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.942555+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.942940+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.943270+0000 mon.c (mon.2) 595 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]: dispatch 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.945062+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]: dispatch 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.947335+0000 mon.c (mon.2) 596 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:39 smithi134 bash[18722]: audit 2024-02-10T08:31:38.948082+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.938192+0000 mon.a (mon.0) 1796 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]': finished 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: cluster 2024-02-10T08:31:38.938341+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.942555+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.942940+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.943270+0000 mon.c (mon.2) 595 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]: dispatch 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.945062+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]: dispatch 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.947335+0000 mon.c (mon.2) 596 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[17385]: audit 2024-02-10T08:31:38.948082+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.938192+0000 mon.a (mon.0) 1796 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-35"}]': finished 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: cluster 2024-02-10T08:31:38.938341+0000 mon.a (mon.0) 1797 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-10T08:31:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.942555+0000 mon.b (mon.1) 77 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.942940+0000 mon.a (mon.0) 1798 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.943270+0000 mon.c (mon.2) 595 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]: dispatch 2024-02-10T08:31:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.945062+0000 mon.a (mon.0) 1799 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]: dispatch 2024-02-10T08:31:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.947335+0000 mon.c (mon.2) 596 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:40.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:39 smithi067 bash[24488]: audit 2024-02-10T08:31:38.948082+0000 mon.a (mon.0) 1800 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout:ete 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6944 ms) 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7050 ms) 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6953 ms) 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7051 ms) 2024-02-10T08:31:40.780 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7036 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7045 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7046 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6836 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7029 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7050 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6935 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7033 ms) 2024-02-10T08:31:40.781 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7047 ms) 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7039 ms) 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6889 ms) 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112052 ms total) 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [----------] Global test environment tear-down 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (204009 ms total) 2024-02-10T08:31:40.782 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio: [ PASSED ] 41 tests. 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54288 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54288 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54647 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54647 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54588 2024-02-10T08:31:40.783 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54588 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54248 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54248 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54348 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54348 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54521 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54521 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54320 2024-02-10T08:31:40.784 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54320 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: cluster 2024-02-10T08:31:39.936139+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.944405+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.944545+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]': finished 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.944673+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: cluster 2024-02-10T08:31:39.944787+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.946342+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.946564+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.950157+0000 mon.c (mon.2) 597 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:39.957238+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: cluster 2024-02-10T08:31:40.001392+0000 mgr.y (mgr.14547) 373 : cluster [DBG] pgmap v460: 228 pgs: 25 creating+peering, 7 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.026980+0000 mon.c (mon.2) 598 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.027595+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.752037+0000 mon.c (mon.2) 599 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.763963+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.764269+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.764386+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: cluster 2024-02-10T08:31:40.764478+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.772816+0000 mon.c (mon.2) 600 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.776356+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.798439+0000 mon.c (mon.2) 601 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.799168+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.800645+0000 mon.c (mon.2) 602 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.801207+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.802011+0000 mon.c (mon.2) 603 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:41.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:40 smithi134 bash[18722]: audit 2024-02-10T08:31:40.802546+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: cluster 2024-02-10T08:31:39.936139+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.944405+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.944545+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]': finished 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.944673+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: cluster 2024-02-10T08:31:39.944787+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.946342+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.946564+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.950157+0000 mon.c (mon.2) 597 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:39.957238+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: cluster 2024-02-10T08:31:40.001392+0000 mgr.y (mgr.14547) 373 : cluster [DBG] pgmap v460: 228 pgs: 25 creating+peering, 7 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.026980+0000 mon.c (mon.2) 598 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.027595+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.752037+0000 mon.c (mon.2) 599 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.763963+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.764269+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.764386+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: cluster 2024-02-10T08:31:40.764478+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: cluster 2024-02-10T08:31:39.936139+0000 mon.a (mon.0) 1801 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:41.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.944405+0000 mon.a (mon.0) 1802 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.944545+0000 mon.a (mon.0) 1803 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-35", "mode": "writeback"}]': finished 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.944673+0000 mon.a (mon.0) 1804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: cluster 2024-02-10T08:31:39.944787+0000 mon.a (mon.0) 1805 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.946342+0000 mon.b (mon.1) 78 : audit [INF] from='client.? 172.21.15.67:0/48207190' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.946564+0000 mon.a (mon.0) 1806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.950157+0000 mon.c (mon.2) 597 : audit [INF] from='client.? 172.21.15.67:0/1794307546' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:39.957238+0000 mon.a (mon.0) 1807 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: cluster 2024-02-10T08:31:40.001392+0000 mgr.y (mgr.14547) 373 : cluster [DBG] pgmap v460: 228 pgs: 25 creating+peering, 7 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.026980+0000 mon.c (mon.2) 598 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.027595+0000 mon.a (mon.0) 1808 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.752037+0000 mon.c (mon.2) 599 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.763963+0000 mon.a (mon.0) 1809 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi067-54245-51"}]': finished 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.764269+0000 mon.a (mon.0) 1810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi067-54237-41"}]': finished 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.764386+0000 mon.a (mon.0) 1811 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:41.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: cluster 2024-02-10T08:31:40.764478+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.772816+0000 mon.c (mon.2) 600 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.776356+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.798439+0000 mon.c (mon.2) 601 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.799168+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.800645+0000 mon.c (mon.2) 602 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.801207+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.802011+0000 mon.c (mon.2) 603 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:41.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[17385]: audit 2024-02-10T08:31:40.802546+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.772816+0000 mon.c (mon.2) 600 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.776356+0000 mon.a (mon.0) 1813 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.798439+0000 mon.c (mon.2) 601 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.799168+0000 mon.a (mon.0) 1814 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.800645+0000 mon.c (mon.2) 602 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.801207+0000 mon.a (mon.0) 1815 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.802011+0000 mon.c (mon.2) 603 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:41.375 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:40 smithi067 bash[24488]: audit 2024-02-10T08:31:40.802546+0000 mon.a (mon.0) 1816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:42.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:31:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.056323+0000 mon.c (mon.2) 604 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.058173+0000 mon.c (mon.2) 605 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.066310+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: cluster 2024-02-10T08:31:41.773456+0000 mon.a (mon.0) 1818 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.781907+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]': finished 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.782211+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: cluster 2024-02-10T08:31:41.782342+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.785788+0000 mon.c (mon.2) 606 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:42.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:41 smithi134 bash[18722]: audit 2024-02-10T08:31:41.786353+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.056323+0000 mon.c (mon.2) 604 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.058173+0000 mon.c (mon.2) 605 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.066310+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: cluster 2024-02-10T08:31:41.773456+0000 mon.a (mon.0) 1818 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.781907+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]': finished 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.782211+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: cluster 2024-02-10T08:31:41.782342+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.785788+0000 mon.c (mon.2) 606 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[24488]: audit 2024-02-10T08:31:41.786353+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.056323+0000 mon.c (mon.2) 604 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:31:42.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.058173+0000 mon.c (mon.2) 605 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.066310+0000 mon.a (mon.0) 1817 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: cluster 2024-02-10T08:31:41.773456+0000 mon.a (mon.0) 1818 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.781907+0000 mon.a (mon.0) 1819 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-35"}]': finished 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.782211+0000 mon.a (mon.0) 1820 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: cluster 2024-02-10T08:31:41.782342+0000 mon.a (mon.0) 1821 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.785788+0000 mon.c (mon.2) 606 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:42.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:41 smithi067 bash[17385]: audit 2024-02-10T08:31:41.786353+0000 mon.a (mon.0) 1822 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:31:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:31:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:43 smithi067 bash[17385]: cluster 2024-02-10T08:31:42.002215+0000 mgr.y (mgr.14547) 374 : cluster [DBG] pgmap v463: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T08:31:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:43 smithi067 bash[17385]: audit 2024-02-10T08:31:42.015122+0000 mgr.y (mgr.14547) 375 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:43 smithi067 bash[17385]: cluster 2024-02-10T08:31:43.234809+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-10T08:31:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:43 smithi067 bash[24488]: cluster 2024-02-10T08:31:42.002215+0000 mgr.y (mgr.14547) 374 : cluster [DBG] pgmap v463: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T08:31:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:43 smithi067 bash[24488]: audit 2024-02-10T08:31:42.015122+0000 mgr.y (mgr.14547) 375 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:43 smithi067 bash[24488]: cluster 2024-02-10T08:31:43.234809+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-10T08:31:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:43 smithi134 bash[18722]: cluster 2024-02-10T08:31:42.002215+0000 mgr.y (mgr.14547) 374 : cluster [DBG] pgmap v463: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-10T08:31:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:43 smithi134 bash[18722]: audit 2024-02-10T08:31:42.015122+0000 mgr.y (mgr.14547) 375 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:43 smithi134 bash[18722]: cluster 2024-02-10T08:31:43.234809+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-10T08:31:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[17385]: audit 2024-02-10T08:31:43.790637+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[17385]: cluster 2024-02-10T08:31:43.790886+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[17385]: audit 2024-02-10T08:31:43.797193+0000 mon.c (mon.2) 607 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[17385]: audit 2024-02-10T08:31:43.806963+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[17385]: audit 2024-02-10T08:31:44.792733+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[17385]: cluster 2024-02-10T08:31:44.792937+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[24488]: audit 2024-02-10T08:31:43.790637+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[24488]: cluster 2024-02-10T08:31:43.790886+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[24488]: audit 2024-02-10T08:31:43.797193+0000 mon.c (mon.2) 607 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[24488]: audit 2024-02-10T08:31:43.806963+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[24488]: audit 2024-02-10T08:31:44.792733+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:45.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:44 smithi067 bash[24488]: cluster 2024-02-10T08:31:44.792937+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-10T08:31:45.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:44 smithi134 bash[18722]: audit 2024-02-10T08:31:43.790637+0000 mon.a (mon.0) 1824 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi067-54245-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:44 smithi134 bash[18722]: cluster 2024-02-10T08:31:43.790886+0000 mon.a (mon.0) 1825 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-10T08:31:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:44 smithi134 bash[18722]: audit 2024-02-10T08:31:43.797193+0000 mon.c (mon.2) 607 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:44 smithi134 bash[18722]: audit 2024-02-10T08:31:43.806963+0000 mon.a (mon.0) 1826 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:44 smithi134 bash[18722]: audit 2024-02-10T08:31:44.792733+0000 mon.a (mon.0) 1827 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:44 smithi134 bash[18722]: cluster 2024-02-10T08:31:44.792937+0000 mon.a (mon.0) 1828 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-10T08:31:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[17385]: cluster 2024-02-10T08:31:44.003226+0000 mgr.y (mgr.14547) 376 : cluster [DBG] pgmap v466: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:31:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[17385]: audit 2024-02-10T08:31:44.816139+0000 mon.c (mon.2) 608 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[17385]: audit 2024-02-10T08:31:44.828599+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[17385]: audit 2024-02-10T08:31:45.775714+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[24488]: cluster 2024-02-10T08:31:44.003226+0000 mgr.y (mgr.14547) 376 : cluster [DBG] pgmap v466: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:31:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[24488]: audit 2024-02-10T08:31:44.816139+0000 mon.c (mon.2) 608 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[24488]: audit 2024-02-10T08:31:44.828599+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:45 smithi067 bash[24488]: audit 2024-02-10T08:31:45.775714+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:46.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:45 smithi134 bash[18722]: cluster 2024-02-10T08:31:44.003226+0000 mgr.y (mgr.14547) 376 : cluster [DBG] pgmap v466: 236 pgs: 40 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:31:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:45 smithi134 bash[18722]: audit 2024-02-10T08:31:44.816139+0000 mon.c (mon.2) 608 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:45 smithi134 bash[18722]: audit 2024-02-10T08:31:44.828599+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:45 smithi134 bash[18722]: audit 2024-02-10T08:31:45.775714+0000 mon.a (mon.0) 1830 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:45.809888+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: cluster 2024-02-10T08:31:45.810086+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:45.811462+0000 mon.c (mon.2) 609 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:45.812535+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:45.812802+0000 mon.c (mon.2) 610 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:45.820248+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: cluster 2024-02-10T08:31:46.004018+0000 mgr.y (mgr.14547) 377 : cluster [DBG] pgmap v469: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:46.812572+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]': finished 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:46.812717+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: cluster 2024-02-10T08:31:46.812834+0000 mon.a (mon.0) 1837 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-10T08:31:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[17385]: audit 2024-02-10T08:31:46.814677+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]: dispatch 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:45.809888+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: cluster 2024-02-10T08:31:45.810086+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:45.811462+0000 mon.c (mon.2) 609 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:45.812535+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:45.812802+0000 mon.c (mon.2) 610 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:45.820248+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: cluster 2024-02-10T08:31:46.004018+0000 mgr.y (mgr.14547) 377 : cluster [DBG] pgmap v469: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:46.812572+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]': finished 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:46.812717+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: cluster 2024-02-10T08:31:46.812834+0000 mon.a (mon.0) 1837 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-10T08:31:47.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:46 smithi067 bash[24488]: audit 2024-02-10T08:31:46.814677+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]: dispatch 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:45.809888+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: cluster 2024-02-10T08:31:45.810086+0000 mon.a (mon.0) 1832 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:45.811462+0000 mon.c (mon.2) 609 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:45.812535+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:45.812802+0000 mon.c (mon.2) 610 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:45.820248+0000 mon.a (mon.0) 1834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: cluster 2024-02-10T08:31:46.004018+0000 mgr.y (mgr.14547) 377 : cluster [DBG] pgmap v469: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:46.812572+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-37"}]': finished 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:46.812717+0000 mon.a (mon.0) 1836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: cluster 2024-02-10T08:31:46.812834+0000 mon.a (mon.0) 1837 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-10T08:31:47.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:46 smithi134 bash[18722]: audit 2024-02-10T08:31:46.814677+0000 mon.a (mon.0) 1838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]: dispatch 2024-02-10T08:31:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:47] "GET /metrics HTTP/1.1" 200 190013 "" "Prometheus/2.33.4" 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: audit 2024-02-10T08:31:46.814003+0000 mon.c (mon.2) 611 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]: dispatch 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: audit 2024-02-10T08:31:46.814469+0000 mon.c (mon.2) 612 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: audit 2024-02-10T08:31:46.816345+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: cluster 2024-02-10T08:31:47.806727+0000 mon.a (mon.0) 1840 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: audit 2024-02-10T08:31:47.816119+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]': finished 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: audit 2024-02-10T08:31:47.816280+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[17385]: cluster 2024-02-10T08:31:47.816377+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: audit 2024-02-10T08:31:46.814003+0000 mon.c (mon.2) 611 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]: dispatch 2024-02-10T08:31:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: audit 2024-02-10T08:31:46.814469+0000 mon.c (mon.2) 612 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: audit 2024-02-10T08:31:46.816345+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: cluster 2024-02-10T08:31:47.806727+0000 mon.a (mon.0) 1840 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: audit 2024-02-10T08:31:47.816119+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]': finished 2024-02-10T08:31:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: audit 2024-02-10T08:31:47.816280+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:48.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:47 smithi067 bash[24488]: cluster 2024-02-10T08:31:47.816377+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-10T08:31:48.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: audit 2024-02-10T08:31:46.814003+0000 mon.c (mon.2) 611 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]: dispatch 2024-02-10T08:31:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: audit 2024-02-10T08:31:46.814469+0000 mon.c (mon.2) 612 : audit [INF] from='client.? 172.21.15.67:0/3564061112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: audit 2024-02-10T08:31:46.816345+0000 mon.a (mon.0) 1839 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]: dispatch 2024-02-10T08:31:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: cluster 2024-02-10T08:31:47.806727+0000 mon.a (mon.0) 1840 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:31:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: audit 2024-02-10T08:31:47.816119+0000 mon.a (mon.0) 1841 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-37", "mode": "writeback"}]': finished 2024-02-10T08:31:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: audit 2024-02-10T08:31:47.816280+0000 mon.a (mon.0) 1842 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi067-54245-52"}]': finished 2024-02-10T08:31:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:47 smithi134 bash[18722]: cluster 2024-02-10T08:31:47.816377+0000 mon.a (mon.0) 1843 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.844584+0000 mon.c (mon.2) 613 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.845050+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.846010+0000 mon.c (mon.2) 614 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.846396+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.847008+0000 mon.c (mon.2) 615 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.847376+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.952085+0000 mon.c (mon.2) 616 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: audit 2024-02-10T08:31:47.952706+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[17385]: cluster 2024-02-10T08:31:48.004867+0000 mgr.y (mgr.14547) 378 : cluster [DBG] pgmap v472: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.844584+0000 mon.c (mon.2) 613 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.845050+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.846010+0000 mon.c (mon.2) 614 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.846396+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.847008+0000 mon.c (mon.2) 615 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.847376+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.952085+0000 mon.c (mon.2) 616 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: audit 2024-02-10T08:31:47.952706+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:49.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:48 smithi067 bash[24488]: cluster 2024-02-10T08:31:48.004867+0000 mgr.y (mgr.14547) 378 : cluster [DBG] pgmap v472: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.844584+0000 mon.c (mon.2) 613 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.845050+0000 mon.a (mon.0) 1844 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.846010+0000 mon.c (mon.2) 614 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.846396+0000 mon.a (mon.0) 1845 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.847008+0000 mon.c (mon.2) 615 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.847376+0000 mon.a (mon.0) 1846 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.952085+0000 mon.c (mon.2) 616 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: audit 2024-02-10T08:31:47.952706+0000 mon.a (mon.0) 1847 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:49.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:48 smithi134 bash[18722]: cluster 2024-02-10T08:31:48.004867+0000 mgr.y (mgr.14547) 378 : cluster [DBG] pgmap v472: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: audit 2024-02-10T08:31:48.835826+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: audit 2024-02-10T08:31:48.836042+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: cluster 2024-02-10T08:31:48.836200+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: audit 2024-02-10T08:31:48.838654+0000 mon.c (mon.2) 617 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: audit 2024-02-10T08:31:48.839129+0000 mon.c (mon.2) 618 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: audit 2024-02-10T08:31:48.841641+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: audit 2024-02-10T08:31:48.841946+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[17385]: cluster 2024-02-10T08:31:49.832077+0000 mon.a (mon.0) 1853 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: audit 2024-02-10T08:31:48.835826+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: audit 2024-02-10T08:31:48.836042+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: cluster 2024-02-10T08:31:48.836200+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: audit 2024-02-10T08:31:48.838654+0000 mon.c (mon.2) 617 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: audit 2024-02-10T08:31:48.839129+0000 mon.c (mon.2) 618 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:50.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: audit 2024-02-10T08:31:48.841641+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:50.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: audit 2024-02-10T08:31:48.841946+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:50.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:50 smithi067 bash[24488]: cluster 2024-02-10T08:31:49.832077+0000 mon.a (mon.0) 1853 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:50.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: audit 2024-02-10T08:31:48.835826+0000 mon.a (mon.0) 1848 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi067-54245-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: audit 2024-02-10T08:31:48.836042+0000 mon.a (mon.0) 1849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: cluster 2024-02-10T08:31:48.836200+0000 mon.a (mon.0) 1850 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: audit 2024-02-10T08:31:48.838654+0000 mon.c (mon.2) 617 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: audit 2024-02-10T08:31:48.839129+0000 mon.c (mon.2) 618 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: audit 2024-02-10T08:31:48.841641+0000 mon.a (mon.0) 1851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]: dispatch 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: audit 2024-02-10T08:31:48.841946+0000 mon.a (mon.0) 1852 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:50 smithi134 bash[18722]: cluster 2024-02-10T08:31:49.832077+0000 mon.a (mon.0) 1853 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:31:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[17385]: audit 2024-02-10T08:31:49.842059+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]': finished 2024-02-10T08:31:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[17385]: cluster 2024-02-10T08:31:50.005641+0000 mgr.y (mgr.14547) 379 : cluster [DBG] pgmap v475: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[17385]: cluster 2024-02-10T08:31:50.293957+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-10T08:31:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[17385]: audit 2024-02-10T08:31:50.844368+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:51.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[17385]: cluster 2024-02-10T08:31:50.844583+0000 mon.a (mon.0) 1857 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-10T08:31:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[24488]: audit 2024-02-10T08:31:49.842059+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]': finished 2024-02-10T08:31:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[24488]: cluster 2024-02-10T08:31:50.005641+0000 mgr.y (mgr.14547) 379 : cluster [DBG] pgmap v475: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[24488]: cluster 2024-02-10T08:31:50.293957+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-10T08:31:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[24488]: audit 2024-02-10T08:31:50.844368+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:51 smithi067 bash[24488]: cluster 2024-02-10T08:31:50.844583+0000 mon.a (mon.0) 1857 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-10T08:31:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:51 smithi134 bash[18722]: audit 2024-02-10T08:31:49.842059+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-37"}]': finished 2024-02-10T08:31:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:51 smithi134 bash[18722]: cluster 2024-02-10T08:31:50.005641+0000 mgr.y (mgr.14547) 379 : cluster [DBG] pgmap v475: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:51 smithi134 bash[18722]: cluster 2024-02-10T08:31:50.293957+0000 mon.a (mon.0) 1855 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-10T08:31:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:51 smithi134 bash[18722]: audit 2024-02-10T08:31:50.844368+0000 mon.a (mon.0) 1856 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi067-54245-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:51 smithi134 bash[18722]: cluster 2024-02-10T08:31:50.844583+0000 mon.a (mon.0) 1857 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-10T08:31:52.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:31:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:31:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[24488]: cluster 2024-02-10T08:31:51.854440+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-10T08:31:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[24488]: audit 2024-02-10T08:31:51.857383+0000 mon.c (mon.2) 619 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:53.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[24488]: audit 2024-02-10T08:31:51.858421+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[24488]: cluster 2024-02-10T08:31:52.006683+0000 mgr.y (mgr.14547) 380 : cluster [DBG] pgmap v478: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[24488]: audit 2024-02-10T08:31:52.026615+0000 mgr.y (mgr.14547) 381 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[17385]: cluster 2024-02-10T08:31:51.854440+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[17385]: audit 2024-02-10T08:31:51.857383+0000 mon.c (mon.2) 619 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[17385]: audit 2024-02-10T08:31:51.858421+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[17385]: cluster 2024-02-10T08:31:52.006683+0000 mgr.y (mgr.14547) 380 : cluster [DBG] pgmap v478: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:31:53.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:52 smithi067 bash[17385]: audit 2024-02-10T08:31:52.026615+0000 mgr.y (mgr.14547) 381 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:52 smithi134 bash[18722]: cluster 2024-02-10T08:31:51.854440+0000 mon.a (mon.0) 1858 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-10T08:31:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:52 smithi134 bash[18722]: audit 2024-02-10T08:31:51.857383+0000 mon.c (mon.2) 619 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:52 smithi134 bash[18722]: audit 2024-02-10T08:31:51.858421+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:31:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:52 smithi134 bash[18722]: cluster 2024-02-10T08:31:52.006683+0000 mgr.y (mgr.14547) 380 : cluster [DBG] pgmap v478: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:31:53.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:52 smithi134 bash[18722]: audit 2024-02-10T08:31:52.026615+0000 mgr.y (mgr.14547) 381 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:31:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:31:54.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[18722]: audit 2024-02-10T08:31:52.860927+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[18722]: cluster 2024-02-10T08:31:52.861269+0000 mon.a (mon.0) 1861 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-10T08:31:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[18722]: audit 2024-02-10T08:31:52.864012+0000 mon.c (mon.2) 620 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[18722]: audit 2024-02-10T08:31:52.864601+0000 mon.c (mon.2) 621 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[18722]: audit 2024-02-10T08:31:52.864803+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:53 smithi134 bash[18722]: audit 2024-02-10T08:31:52.865732+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:54.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[17385]: audit 2024-02-10T08:31:52.860927+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[17385]: cluster 2024-02-10T08:31:52.861269+0000 mon.a (mon.0) 1861 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[17385]: audit 2024-02-10T08:31:52.864012+0000 mon.c (mon.2) 620 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[17385]: audit 2024-02-10T08:31:52.864601+0000 mon.c (mon.2) 621 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[17385]: audit 2024-02-10T08:31:52.864803+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[17385]: audit 2024-02-10T08:31:52.865732+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[24488]: audit 2024-02-10T08:31:52.860927+0000 mon.a (mon.0) 1860 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[24488]: cluster 2024-02-10T08:31:52.861269+0000 mon.a (mon.0) 1861 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[24488]: audit 2024-02-10T08:31:52.864012+0000 mon.c (mon.2) 620 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[24488]: audit 2024-02-10T08:31:52.864601+0000 mon.c (mon.2) 621 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[24488]: audit 2024-02-10T08:31:52.864803+0000 mon.a (mon.0) 1862 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:53 smithi067 bash[24488]: audit 2024-02-10T08:31:52.865732+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:53.867366+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:53.867632+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: cluster 2024-02-10T08:31:53.867830+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:53.869939+0000 mon.c (mon.2) 622 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:53.870524+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:53.872194+0000 mon.c (mon.2) 623 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:53.877395+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: cluster 2024-02-10T08:31:54.007696+0000 mgr.y (mgr.14547) 382 : cluster [DBG] pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:54.869231+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:31:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: audit 2024-02-10T08:31:54.869399+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:55.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:54 smithi134 bash[18722]: cluster 2024-02-10T08:31:54.869528+0000 mon.a (mon.0) 1871 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-10T08:31:55.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:53.867366+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:53.867632+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: cluster 2024-02-10T08:31:53.867830+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:53.869939+0000 mon.c (mon.2) 622 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:53.870524+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:53.872194+0000 mon.c (mon.2) 623 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:53.877395+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: cluster 2024-02-10T08:31:54.007696+0000 mgr.y (mgr.14547) 382 : cluster [DBG] pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:54.869231+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: audit 2024-02-10T08:31:54.869399+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[24488]: cluster 2024-02-10T08:31:54.869528+0000 mon.a (mon.0) 1871 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:53.867366+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:53.867632+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: cluster 2024-02-10T08:31:53.867830+0000 mon.a (mon.0) 1866 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:53.869939+0000 mon.c (mon.2) 622 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:53.870524+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:53.872194+0000 mon.c (mon.2) 623 : audit [INF] from='client.? 172.21.15.67:0/497688170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:53.877395+0000 mon.a (mon.0) 1868 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]: dispatch 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: cluster 2024-02-10T08:31:54.007696+0000 mgr.y (mgr.14547) 382 : cluster [DBG] pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:54.869231+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: audit 2024-02-10T08:31:54.869399+0000 mon.a (mon.0) 1870 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi067-54245-53"}]': finished 2024-02-10T08:31:55.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:54 smithi067 bash[17385]: cluster 2024-02-10T08:31:54.869528+0000 mon.a (mon.0) 1871 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-10T08:31:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:55 smithi134 bash[18722]: audit 2024-02-10T08:31:54.874389+0000 mon.c (mon.2) 624 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:31:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:55 smithi134 bash[18722]: audit 2024-02-10T08:31:54.881092+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:31:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:55 smithi134 bash[18722]: audit 2024-02-10T08:31:54.901997+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:55 smithi134 bash[18722]: audit 2024-02-10T08:31:54.902833+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:55 smithi134 bash[18722]: audit 2024-02-10T08:31:54.903402+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi067-54245-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[17385]: audit 2024-02-10T08:31:54.874389+0000 mon.c (mon.2) 624 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:31:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[17385]: audit 2024-02-10T08:31:54.881092+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:31:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[17385]: audit 2024-02-10T08:31:54.901997+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[17385]: audit 2024-02-10T08:31:54.902833+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[17385]: audit 2024-02-10T08:31:54.903402+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi067-54245-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[24488]: audit 2024-02-10T08:31:54.874389+0000 mon.c (mon.2) 624 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[24488]: audit 2024-02-10T08:31:54.881092+0000 mon.a (mon.0) 1872 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[24488]: audit 2024-02-10T08:31:54.901997+0000 mon.a (mon.0) 1873 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[24488]: audit 2024-02-10T08:31:54.902833+0000 mon.a (mon.0) 1874 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:56.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:55 smithi067 bash[24488]: audit 2024-02-10T08:31:54.903402+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi067-54245-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:31:57.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: audit 2024-02-10T08:31:55.892532+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:31:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: audit 2024-02-10T08:31:55.892716+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi067-54245-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: cluster 2024-02-10T08:31:55.892849+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-10T08:31:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: audit 2024-02-10T08:31:55.893895+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi067-54245-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: audit 2024-02-10T08:31:55.898744+0000 mon.c (mon.2) 625 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:31:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: audit 2024-02-10T08:31:55.899409+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:31:57.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:56 smithi134 bash[18722]: cluster 2024-02-10T08:31:56.008567+0000 mgr.y (mgr.14547) 383 : cluster [DBG] pgmap v484: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: audit 2024-02-10T08:31:55.892532+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:31:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: audit 2024-02-10T08:31:55.892716+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi067-54245-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: cluster 2024-02-10T08:31:55.892849+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-10T08:31:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: audit 2024-02-10T08:31:55.893895+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi067-54245-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: audit 2024-02-10T08:31:55.898744+0000 mon.c (mon.2) 625 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: audit 2024-02-10T08:31:55.899409+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[17385]: cluster 2024-02-10T08:31:56.008567+0000 mgr.y (mgr.14547) 383 : cluster [DBG] pgmap v484: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: audit 2024-02-10T08:31:55.892532+0000 mon.a (mon.0) 1876 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: audit 2024-02-10T08:31:55.892716+0000 mon.a (mon.0) 1877 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi067-54245-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: cluster 2024-02-10T08:31:55.892849+0000 mon.a (mon.0) 1878 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: audit 2024-02-10T08:31:55.893895+0000 mon.a (mon.0) 1879 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi067-54245-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: audit 2024-02-10T08:31:55.898744+0000 mon.c (mon.2) 625 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: audit 2024-02-10T08:31:55.899409+0000 mon.a (mon.0) 1880 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:31:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:56 smithi067 bash[24488]: cluster 2024-02-10T08:31:56.008567+0000 mgr.y (mgr.14547) 383 : cluster [DBG] pgmap v484: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:31:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:31:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:31:57] "GET /metrics HTTP/1.1" 200 190013 "" "Prometheus/2.33.4" 2024-02-10T08:31:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:57 smithi067 bash[24488]: audit 2024-02-10T08:31:57.354887+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:31:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:57 smithi067 bash[24488]: cluster 2024-02-10T08:31:57.354994+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-10T08:31:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:57 smithi067 bash[17385]: audit 2024-02-10T08:31:57.354887+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:31:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:57 smithi067 bash[17385]: cluster 2024-02-10T08:31:57.354994+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-10T08:31:58.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:57 smithi134 bash[18722]: audit 2024-02-10T08:31:57.354887+0000 mon.a (mon.0) 1881 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:31:58.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:57 smithi134 bash[18722]: cluster 2024-02-10T08:31:57.354994+0000 mon.a (mon.0) 1882 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: audit 2024-02-10T08:31:57.912967+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi067-54245-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: cluster 2024-02-10T08:31:57.913102+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: cluster 2024-02-10T08:31:58.009400+0000 mgr.y (mgr.14547) 384 : cluster [DBG] pgmap v487: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: audit 2024-02-10T08:31:58.383552+0000 mon.c (mon.2) 626 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: audit 2024-02-10T08:31:58.384130+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: audit 2024-02-10T08:31:58.385043+0000 mon.c (mon.2) 627 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]: dispatch 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: audit 2024-02-10T08:31:58.385557+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]: dispatch 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: audit 2024-02-10T08:31:58.915892+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]': finished 2024-02-10T08:31:59.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:31:58 smithi134 bash[18722]: cluster 2024-02-10T08:31:58.916113+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: audit 2024-02-10T08:31:57.912967+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi067-54245-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: cluster 2024-02-10T08:31:57.913102+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: cluster 2024-02-10T08:31:58.009400+0000 mgr.y (mgr.14547) 384 : cluster [DBG] pgmap v487: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: audit 2024-02-10T08:31:58.383552+0000 mon.c (mon.2) 626 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: audit 2024-02-10T08:31:58.384130+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: audit 2024-02-10T08:31:58.385043+0000 mon.c (mon.2) 627 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]: dispatch 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: audit 2024-02-10T08:31:58.385557+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]: dispatch 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: audit 2024-02-10T08:31:58.915892+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]': finished 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[17385]: cluster 2024-02-10T08:31:58.916113+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-10T08:31:59.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: audit 2024-02-10T08:31:57.912967+0000 mon.a (mon.0) 1883 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi067-54245-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: cluster 2024-02-10T08:31:57.913102+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: cluster 2024-02-10T08:31:58.009400+0000 mgr.y (mgr.14547) 384 : cluster [DBG] pgmap v487: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: audit 2024-02-10T08:31:58.383552+0000 mon.c (mon.2) 626 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: audit 2024-02-10T08:31:58.384130+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: audit 2024-02-10T08:31:58.385043+0000 mon.c (mon.2) 627 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]: dispatch 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: audit 2024-02-10T08:31:58.385557+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]: dispatch 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: audit 2024-02-10T08:31:58.915892+0000 mon.a (mon.0) 1887 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-39"}]': finished 2024-02-10T08:31:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:31:58 smithi067 bash[24488]: cluster 2024-02-10T08:31:58.916113+0000 mon.a (mon.0) 1888 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [==========] Running 72 tests from 4 test suites. 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: seed 54339 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (12 ms) 2024-02-10T08:31:59.933 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (16637 ms) 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (62 ms) 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16711 ms total) 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7602 ms) 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (9677 ms) 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9995 ms) 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-10T08:31:59.934 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: my_snaps [4] 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: my_snaps [5,4] 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: my_snaps [6,5,4] 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: promoting some heads 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: promoting from clones for snap 7 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: waiting for scrubs... 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: done waiting 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (47041 ms) 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10084 ms) 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-10T08:31:59.935 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7936 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8066 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8059 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10499 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8379 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10073 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7941 ms) 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-10T08:31:59.936 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8491 ms) 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (12662 ms) 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (8309 ms) 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (7612 ms) 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-10T08:31:59.937 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: ok, hit_set contains 189:602f83fe:::foo:head 2024-02-10T08:32:01.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: cluster 2024-02-10T08:31:59.930554+0000 mon.a (mon.0) 1889 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-10T08:32:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: audit 2024-02-10T08:31:59.933779+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:32:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: cluster 2024-02-10T08:32:00.010181+0000 mgr.y (mgr.14547) 385 : cluster [DBG] pgmap v490: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: audit 2024-02-10T08:32:00.886463+0000 mon.c (mon.2) 628 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]: dispatch 2024-02-10T08:32:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: audit 2024-02-10T08:32:00.887075+0000 mon.c (mon.2) 629 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-10T08:32:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: audit 2024-02-10T08:32:00.887458+0000 mon.a (mon.0) 1891 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]: dispatch 2024-02-10T08:32:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:00 smithi134 bash[18722]: audit 2024-02-10T08:32:00.888201+0000 mon.a (mon.0) 1892 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-10T08:32:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: cluster 2024-02-10T08:31:59.930554+0000 mon.a (mon.0) 1889 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-10T08:32:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: audit 2024-02-10T08:31:59.933779+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:32:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: cluster 2024-02-10T08:32:00.010181+0000 mgr.y (mgr.14547) 385 : cluster [DBG] pgmap v490: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: audit 2024-02-10T08:32:00.886463+0000 mon.c (mon.2) 628 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: audit 2024-02-10T08:32:00.887075+0000 mon.c (mon.2) 629 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: audit 2024-02-10T08:32:00.887458+0000 mon.a (mon.0) 1891 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[17385]: audit 2024-02-10T08:32:00.888201+0000 mon.a (mon.0) 1892 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: cluster 2024-02-10T08:31:59.930554+0000 mon.a (mon.0) 1889 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: audit 2024-02-10T08:31:59.933779+0000 mon.a (mon.0) 1890 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: cluster 2024-02-10T08:32:00.010181+0000 mgr.y (mgr.14547) 385 : cluster [DBG] pgmap v490: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: audit 2024-02-10T08:32:00.886463+0000 mon.c (mon.2) 628 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: audit 2024-02-10T08:32:00.887075+0000 mon.c (mon.2) 629 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: audit 2024-02-10T08:32:00.887458+0000 mon.a (mon.0) 1891 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]: dispatch 2024-02-10T08:32:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:00 smithi067 bash[24488]: audit 2024-02-10T08:32:00.888201+0000 mon.a (mon.0) 1892 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-10T08:32:02.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: audit 2024-02-10T08:32:00.937979+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: audit 2024-02-10T08:32:00.938206+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]': finished 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: audit 2024-02-10T08:32:00.938352+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]': finished 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: cluster 2024-02-10T08:32:00.938454+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: audit 2024-02-10T08:32:00.940583+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: audit 2024-02-10T08:32:00.942736+0000 mon.c (mon.2) 630 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:01 smithi134 bash[18722]: audit 2024-02-10T08:32:00.949094+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: audit 2024-02-10T08:32:00.937979+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:32:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: audit 2024-02-10T08:32:00.938206+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]': finished 2024-02-10T08:32:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: audit 2024-02-10T08:32:00.938352+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]': finished 2024-02-10T08:32:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: cluster 2024-02-10T08:32:00.938454+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-10T08:32:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: audit 2024-02-10T08:32:00.940583+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:32:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: audit 2024-02-10T08:32:00.942736+0000 mon.c (mon.2) 630 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[17385]: audit 2024-02-10T08:32:00.949094+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: audit 2024-02-10T08:32:00.937979+0000 mon.a (mon.0) 1893 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: audit 2024-02-10T08:32:00.938206+0000 mon.a (mon.0) 1894 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.f", "id": [1, 7]}]': finished 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: audit 2024-02-10T08:32:00.938352+0000 mon.a (mon.0) 1895 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]': finished 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: cluster 2024-02-10T08:32:00.938454+0000 mon.a (mon.0) 1896 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: audit 2024-02-10T08:32:00.940583+0000 mon.a (mon.0) 1897 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]: dispatch 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: audit 2024-02-10T08:32:00.942736+0000 mon.c (mon.2) 630 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:01 smithi067 bash[24488]: audit 2024-02-10T08:32:00.949094+0000 mon.a (mon.0) 1898 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:03.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:32:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:32:03.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.941706+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:32:03.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.942378+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:03.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: cluster 2024-02-10T08:32:01.942607+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-10T08:32:03.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.976540+0000 mon.c (mon.2) 631 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi067-54339-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T08:32:03.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.977131+0000 mon.c (mon.2) 632 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.977556+0000 mon.a (mon.0) 1902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.980625+0000 mon.c (mon.2) 633 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.981003+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.981655+0000 mon.c (mon.2) 634 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.982049+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.982561+0000 mon.c (mon.2) 635 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:01.982926+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: cluster 2024-02-10T08:32:02.010989+0000 mgr.y (mgr.14547) 386 : cluster [DBG] pgmap v493: 228 pgs: 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.035014+0000 mgr.y (mgr.14547) 387 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.945888+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.946022+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: cluster 2024-02-10T08:32:02.946125+0000 mon.a (mon.0) 1908 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.948243+0000 mon.c (mon.2) 636 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.948721+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.953792+0000 mon.c (mon.2) 637 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T08:32:03.279 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:02 smithi134 bash[18722]: audit 2024-02-10T08:32:02.954220+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T08:32:03.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.941706+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.942378+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: cluster 2024-02-10T08:32:01.942607+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.976540+0000 mon.c (mon.2) 631 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi067-54339-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.977131+0000 mon.c (mon.2) 632 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.977556+0000 mon.a (mon.0) 1902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.980625+0000 mon.c (mon.2) 633 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.981003+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.981655+0000 mon.c (mon.2) 634 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.982049+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.982561+0000 mon.c (mon.2) 635 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:01.982926+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: cluster 2024-02-10T08:32:02.010989+0000 mgr.y (mgr.14547) 386 : cluster [DBG] pgmap v493: 228 pgs: 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.035014+0000 mgr.y (mgr.14547) 387 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.945888+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.946022+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: cluster 2024-02-10T08:32:02.946125+0000 mon.a (mon.0) 1908 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.948243+0000 mon.c (mon.2) 636 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.948721+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.953792+0000 mon.c (mon.2) 637 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[17385]: audit 2024-02-10T08:32:02.954220+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.941706+0000 mon.a (mon.0) 1899 : audit [INF] from='client.? 172.21.15.67:0/2701603923' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi067-54245-54"}]': finished 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.942378+0000 mon.a (mon.0) 1900 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: cluster 2024-02-10T08:32:01.942607+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-10T08:32:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.976540+0000 mon.c (mon.2) 631 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi067-54339-6","var": "pg_num","format": "json"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.977131+0000 mon.c (mon.2) 632 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.977556+0000 mon.a (mon.0) 1902 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.980625+0000 mon.c (mon.2) 633 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.981003+0000 mon.a (mon.0) 1903 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.981655+0000 mon.c (mon.2) 634 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.982049+0000 mon.a (mon.0) 1904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.982561+0000 mon.c (mon.2) 635 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:01.982926+0000 mon.a (mon.0) 1905 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: cluster 2024-02-10T08:32:02.010989+0000 mgr.y (mgr.14547) 386 : cluster [DBG] pgmap v493: 228 pgs: 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.035014+0000 mgr.y (mgr.14547) 387 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.945888+0000 mon.a (mon.0) 1906 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.946022+0000 mon.a (mon.0) 1907 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi067-54245-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: cluster 2024-02-10T08:32:02.946125+0000 mon.a (mon.0) 1908 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.948243+0000 mon.c (mon.2) 636 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.948721+0000 mon.a (mon.0) 1909 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:03.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.953792+0000 mon.c (mon.2) 637 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T08:32:03.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:02 smithi067 bash[24488]: audit 2024-02-10T08:32:02.954220+0000 mon.a (mon.0) 1910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-10T08:32:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:04 smithi134 bash[18722]: cluster 2024-02-10T08:32:04.011759+0000 mgr.y (mgr.14547) 388 : cluster [DBG] pgmap v496: 228 pgs: 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:04 smithi134 bash[18722]: audit 2024-02-10T08:32:04.398387+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]': finished 2024-02-10T08:32:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:04 smithi134 bash[18722]: cluster 2024-02-10T08:32:04.398631+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-10T08:32:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:04 smithi134 bash[18722]: audit 2024-02-10T08:32:04.399542+0000 mon.c (mon.2) 638 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:04 smithi134 bash[18722]: audit 2024-02-10T08:32:04.402471+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[24488]: cluster 2024-02-10T08:32:04.011759+0000 mgr.y (mgr.14547) 388 : cluster [DBG] pgmap v496: 228 pgs: 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[24488]: audit 2024-02-10T08:32:04.398387+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]': finished 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[24488]: cluster 2024-02-10T08:32:04.398631+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[24488]: audit 2024-02-10T08:32:04.399542+0000 mon.c (mon.2) 638 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[24488]: audit 2024-02-10T08:32:04.402471+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[17385]: cluster 2024-02-10T08:32:04.011759+0000 mgr.y (mgr.14547) 388 : cluster [DBG] pgmap v496: 228 pgs: 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[17385]: audit 2024-02-10T08:32:04.398387+0000 mon.a (mon.0) 1911 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_count","val": "8"}]': finished 2024-02-10T08:32:05.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[17385]: cluster 2024-02-10T08:32:04.398631+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-10T08:32:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[17385]: audit 2024-02-10T08:32:04.399542+0000 mon.c (mon.2) 638 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:04 smithi067 bash[17385]: audit 2024-02-10T08:32:04.402471+0000 mon.a (mon.0) 1913 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:05 smithi134 bash[18722]: audit 2024-02-10T08:32:04.958739+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:05 smithi134 bash[18722]: audit 2024-02-10T08:32:04.958907+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:32:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:05 smithi134 bash[18722]: cluster 2024-02-10T08:32:04.959008+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-10T08:32:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:05 smithi134 bash[18722]: audit 2024-02-10T08:32:04.961104+0000 mon.c (mon.2) 639 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T08:32:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:05 smithi134 bash[18722]: audit 2024-02-10T08:32:04.961762+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T08:32:06.359 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9085 ms) 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg_num = 32 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 0 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 1 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 2 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 3 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 4 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 5 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 6 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 7 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 8 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 9 ls 1707553926,0 2024-02-10T08:32:06.360 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 10 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 11 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 12 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 13 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 14 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 15 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 16 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 17 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 18 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 19 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 20 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 21 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 22 ls 1707553926,0 2024-02-10T08:32:06.361 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 23 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 24 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 25 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 26 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 27 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 28 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 29 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 30 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg 31 ls 1707553926,0 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: pg_num = 32 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6cac518f:::0:head 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:02547ec2:::1:head 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f905c69b:::2:head 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:cfc208b3:::3:head 2024-02-10T08:32:06.362 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d83876eb:::4:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b29083e3:::5:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c4fdafeb:::6:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:5c6b0b28:::7:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:bd63b0f1:::8:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e960b815:::9:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:52ea6a34:::10:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:89d3ae78:::11:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:de5d7c5f:::12:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:566253c9:::13:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:62a1935d:::14:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:863748b0:::15:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3958e169:::16:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4d4dabf9:::17:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:8391935d:::18:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:28883081:::19:head 2024-02-10T08:32:06.363 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:69259c59:::20:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4bdb80b7:::21:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a11c5d71:::22:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:271af37b:::23:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:95b121be:::24:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:58d1031b:::25:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:0a050783:::26:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c709704c:::27:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:cbe56eaf:::28:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:86b4b162:::29:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:70d89383:::30:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:dd450c7c:::31:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6d5729b1:::32:head 2024-02-10T08:32:06.364 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c388f3fb:::33:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:56cfea31:::34:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9dbc1bf7:::35:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:40b74ccd:::36:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4d5aaf42:::37:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:920f362c:::38:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6cc53222:::39:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9cad833f:::40:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:1ea84d41:::41:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c4480ef6:::42:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a694361e:::43:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d1bd33e9:::44:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ddc2cd5d:::45:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:2b782207:::46:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7b187fca:::47:head 2024-02-10T08:32:06.365 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:90ecdf6f:::48:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a5ed95fe:::49:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ea0eaa55:::50:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f33ef17b:::51:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a0d1b2f6:::52:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:60c5229e:::53:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:edcbc575:::54:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:102cf253:::55:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:efb7fb0b:::56:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:50d0a326:::57:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d4dc5daf:::58:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3a130462:::59:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ec87ed71:::60:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d5bc9454:::61:head 2024-02-10T08:32:06.366 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3ddfe313:::62:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7c2816b9:::63:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:47e00e4d:::64:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c6410c18:::65:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b48ed237:::66:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:cd63ad31:::67:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b179e92b:::68:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:0d9f741a:::69:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6d3352ae:::70:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c6d5c19e:::71:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:bc4729c3:::72:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:77e930b9:::73:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:0abeecfd:::74:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b7c37e15:::75:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b6378398:::76:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:02bd68de:::77:head 2024-02-10T08:32:06.367 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:cc795d2d:::78:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:630d4fea:::79:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e0d29ef5:::80:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:fd6f13d2:::81:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:606461d5:::82:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:eadbdc43:::83:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:8761d0bb:::84:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9ef0186f:::85:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e0d41294:::86:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:961de695:::87:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:1423148f:::88:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:633a8fa2:::89:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a8653809:::90:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3dac8b33:::91:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:35aad435:::92:head 2024-02-10T08:32:06.368 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f6dcc343:::93:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:dbbdad87:::94:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:1cb48ce0:::95:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:03cd461c:::96:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:17a4ea99:::97:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9993c9a7:::98:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6394211c:::99:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:94c7ae57:::100:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6fdee5bb:::101:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9a477fd1:::102:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:eb850916:::103:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:affc56b9:::104:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b42dc814:::105:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f319f8f0:::106:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9a40b9de:::107:head 2024-02-10T08:32:06.369 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:8b524f28:::108:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e3de589f:::109:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:90f90a5b:::110:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a7b4f1d7:::111:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:af51766e:::112:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b6f90bd1:::113:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e0261208:::114:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c9569ef7:::115:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:61bebe50:::116:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:fe93412b:::117:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d3d38bee:::118:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3100ba0c:::119:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d0560ada:::120:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f0ea8b35:::121:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:766f231a:::122:head 2024-02-10T08:32:06.370 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a07a2582:::123:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:bd7c6b3a:::124:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:fb2ddaff:::125:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4408e1fe:::126:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ee1df7a7:::127:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c3002909:::128:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4f48ffa9:::129:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:edf38733:::130:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c08425c0:::131:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:5f902d98:::132:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:41ea2c93:::133:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:813cee13:::134:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:0131818d:::135:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:26ba5a85:::136:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:381b8a5a:::137:head 2024-02-10T08:32:06.371 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:28797e47:::138:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:bfca7f22:::139:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:36807075:::140:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:80b03975:::141:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:5c15709b:::142:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f39ea15e:::143:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ea992956:::144:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:48887b1c:::145:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:9f24a9dd:::146:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:987f100b:::147:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d2dd3581:::148:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7fed1808:::149:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c80b70e9:::150:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:85ed90f9:::151:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:36428b24:::152:head 2024-02-10T08:32:06.372 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d044c34a:::153:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7c18bf58:::154:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d1c21232:::155:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a7a3c575:::156:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:87da0633:::157:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d5ac3822:::158:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3f20522d:::159:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6ca26563:::160:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:532ce135:::161:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:c78863e6:::162:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7462ddf6:::163:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:5d165639:::164:head 2024-02-10T08:32:06.373 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f43765fc:::165:head 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[24488]: audit 2024-02-10T08:32:04.958739+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[24488]: audit 2024-02-10T08:32:04.958907+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[24488]: cluster 2024-02-10T08:32:04.959008+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[24488]: audit 2024-02-10T08:32:04.961104+0000 mon.c (mon.2) 639 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[24488]: audit 2024-02-10T08:32:04.961762+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[17385]: audit 2024-02-10T08:32:04.958739+0000 mon.a (mon.0) 1914 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi067-54245-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[17385]: audit 2024-02-10T08:32:04.958907+0000 mon.a (mon.0) 1915 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[17385]: cluster 2024-02-10T08:32:04.959008+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[17385]: audit 2024-02-10T08:32:04.961104+0000 mon.c (mon.2) 639 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T08:32:06.374 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:05 smithi067 bash[17385]: audit 2024-02-10T08:32:04.961762+0000 mon.a (mon.0) 1917 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-10T08:32:07.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: audit 2024-02-10T08:32:05.964920+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: cluster 2024-02-10T08:32:05.965609+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: cluster 2024-02-10T08:32:06.012373+0000 mgr.y (mgr.14547) 389 : cluster [DBG] pgmap v499: 236 pgs: 8 unknown, 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: audit 2024-02-10T08:32:06.358718+0000 mon.c (mon.2) 640 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi067-54339-41","var": "pg_num","format": "json"}]: dispatch 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: audit 2024-02-10T08:32:06.601436+0000 mon.c (mon.2) 641 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: audit 2024-02-10T08:32:06.602154+0000 mon.a (mon.0) 1920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: audit 2024-02-10T08:32:06.603613+0000 mon.c (mon.2) 642 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]: dispatch 2024-02-10T08:32:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:06 smithi134 bash[18722]: audit 2024-02-10T08:32:06.604300+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]: dispatch 2024-02-10T08:32:07.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: audit 2024-02-10T08:32:05.964920+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T08:32:07.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: cluster 2024-02-10T08:32:05.965609+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-10T08:32:07.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: cluster 2024-02-10T08:32:06.012373+0000 mgr.y (mgr.14547) 389 : cluster [DBG] pgmap v499: 236 pgs: 8 unknown, 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: audit 2024-02-10T08:32:06.358718+0000 mon.c (mon.2) 640 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi067-54339-41","var": "pg_num","format": "json"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: audit 2024-02-10T08:32:06.601436+0000 mon.c (mon.2) 641 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: audit 2024-02-10T08:32:06.602154+0000 mon.a (mon.0) 1920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: audit 2024-02-10T08:32:06.603613+0000 mon.c (mon.2) 642 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[24488]: audit 2024-02-10T08:32:06.604300+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: audit 2024-02-10T08:32:05.964920+0000 mon.a (mon.0) 1918 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: cluster 2024-02-10T08:32:05.965609+0000 mon.a (mon.0) 1919 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: cluster 2024-02-10T08:32:06.012373+0000 mgr.y (mgr.14547) 389 : cluster [DBG] pgmap v499: 236 pgs: 8 unknown, 32 creating+peering, 2 peering, 194 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: audit 2024-02-10T08:32:06.358718+0000 mon.c (mon.2) 640 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi067-54339-41","var": "pg_num","format": "json"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: audit 2024-02-10T08:32:06.601436+0000 mon.c (mon.2) 641 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:07.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: audit 2024-02-10T08:32:06.602154+0000 mon.a (mon.0) 1920 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: audit 2024-02-10T08:32:06.603613+0000 mon.c (mon.2) 642 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]: dispatch 2024-02-10T08:32:07.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:06 smithi067 bash[17385]: audit 2024-02-10T08:32:06.604300+0000 mon.a (mon.0) 1921 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]: dispatch 2024-02-10T08:32:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:32:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:07] "GET /metrics HTTP/1.1" 200 185191 "" "Prometheus/2.33.4" 2024-02-10T08:32:08.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:08 smithi134 bash[18722]: audit 2024-02-10T08:32:06.988455+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]': finished 2024-02-10T08:32:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:08 smithi134 bash[18722]: cluster 2024-02-10T08:32:06.988581+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-10T08:32:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:08 smithi134 bash[18722]: audit 2024-02-10T08:32:06.995048+0000 mon.c (mon.2) 643 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:08 smithi134 bash[18722]: audit 2024-02-10T08:32:06.996759+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[24488]: audit 2024-02-10T08:32:06.988455+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]': finished 2024-02-10T08:32:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[24488]: cluster 2024-02-10T08:32:06.988581+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-10T08:32:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[24488]: audit 2024-02-10T08:32:06.995048+0000 mon.c (mon.2) 643 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[24488]: audit 2024-02-10T08:32:06.996759+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[17385]: audit 2024-02-10T08:32:06.988455+0000 mon.a (mon.0) 1922 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-41"}]': finished 2024-02-10T08:32:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[17385]: cluster 2024-02-10T08:32:06.988581+0000 mon.a (mon.0) 1923 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-10T08:32:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[17385]: audit 2024-02-10T08:32:06.995048+0000 mon.c (mon.2) 643 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:08 smithi067 bash[17385]: audit 2024-02-10T08:32:06.996759+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:07.994475+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: cluster 2024-02-10T08:32:07.994622+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-10T08:32:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:07.996392+0000 mon.c (mon.2) 644 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:08.001500+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: cluster 2024-02-10T08:32:08.012906+0000 mgr.y (mgr.14547) 390 : cluster [DBG] pgmap v502: 196 pgs: 1 active+undersized+remapped, 195 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 32 B/s, 13 objects/s recovering 2024-02-10T08:32:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:08.998377+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: cluster 2024-02-10T08:32:08.998545+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.000489+0000 mon.c (mon.2) 645 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.001585+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.024964+0000 mon.c (mon.2) 646 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.025459+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.026304+0000 mon.c (mon.2) 647 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.026815+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.027600+0000 mon.c (mon.2) 648 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:09 smithi134 bash[18722]: audit 2024-02-10T08:32:09.028077+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:07.994475+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: cluster 2024-02-10T08:32:07.994622+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:07.996392+0000 mon.c (mon.2) 644 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:08.001500+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: cluster 2024-02-10T08:32:08.012906+0000 mgr.y (mgr.14547) 390 : cluster [DBG] pgmap v502: 196 pgs: 1 active+undersized+remapped, 195 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 32 B/s, 13 objects/s recovering 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:08.998377+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: cluster 2024-02-10T08:32:08.998545+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.000489+0000 mon.c (mon.2) 645 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.001585+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.024964+0000 mon.c (mon.2) 646 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.025459+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.026304+0000 mon.c (mon.2) 647 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.026815+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.027600+0000 mon.c (mon.2) 648 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[17385]: audit 2024-02-10T08:32:09.028077+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:07.994475+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: cluster 2024-02-10T08:32:07.994622+0000 mon.a (mon.0) 1926 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:07.996392+0000 mon.c (mon.2) 644 : audit [INF] from='client.? 172.21.15.67:0/413065974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:08.001500+0000 mon.a (mon.0) 1927 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]: dispatch 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: cluster 2024-02-10T08:32:08.012906+0000 mgr.y (mgr.14547) 390 : cluster [DBG] pgmap v502: 196 pgs: 1 active+undersized+remapped, 195 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 32 B/s, 13 objects/s recovering 2024-02-10T08:32:09.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:08.998377+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi067-54245-55"}]': finished 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: cluster 2024-02-10T08:32:08.998545+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.000489+0000 mon.c (mon.2) 645 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.001585+0000 mon.a (mon.0) 1930 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.024964+0000 mon.c (mon.2) 646 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.025459+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.026304+0000 mon.c (mon.2) 647 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.026815+0000 mon.a (mon.0) 1932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.027600+0000 mon.c (mon.2) 648 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:09.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:09 smithi067 bash[24488]: audit 2024-02-10T08:32:09.028077+0000 mon.a (mon.0) 1933 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: audit 2024-02-10T08:32:10.003498+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: audit 2024-02-10T08:32:10.004043+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: cluster 2024-02-10T08:32:10.004311+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: audit 2024-02-10T08:32:10.007325+0000 mon.c (mon.2) 649 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: audit 2024-02-10T08:32:10.008465+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: audit 2024-02-10T08:32:10.009864+0000 mon.c (mon.2) 650 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: audit 2024-02-10T08:32:10.011791+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:11.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:11 smithi134 bash[18722]: cluster 2024-02-10T08:32:10.014402+0000 mgr.y (mgr.14547) 391 : cluster [DBG] pgmap v505: 228 pgs: 32 unknown, 1 active+undersized+remapped, 195 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 32 B/s, 13 objects/s recovering 2024-02-10T08:32:11.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: audit 2024-02-10T08:32:10.003498+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:11.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: audit 2024-02-10T08:32:10.004043+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:11.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: cluster 2024-02-10T08:32:10.004311+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: audit 2024-02-10T08:32:10.007325+0000 mon.c (mon.2) 649 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: audit 2024-02-10T08:32:10.008465+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: audit 2024-02-10T08:32:10.009864+0000 mon.c (mon.2) 650 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: audit 2024-02-10T08:32:10.011791+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[24488]: cluster 2024-02-10T08:32:10.014402+0000 mgr.y (mgr.14547) 391 : cluster [DBG] pgmap v505: 228 pgs: 32 unknown, 1 active+undersized+remapped, 195 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 32 B/s, 13 objects/s recovering 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: audit 2024-02-10T08:32:10.003498+0000 mon.a (mon.0) 1934 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: audit 2024-02-10T08:32:10.004043+0000 mon.a (mon.0) 1935 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi067-54245-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: cluster 2024-02-10T08:32:10.004311+0000 mon.a (mon.0) 1936 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: audit 2024-02-10T08:32:10.007325+0000 mon.c (mon.2) 649 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: audit 2024-02-10T08:32:10.008465+0000 mon.a (mon.0) 1937 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: audit 2024-02-10T08:32:10.009864+0000 mon.c (mon.2) 650 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:11.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: audit 2024-02-10T08:32:10.011791+0000 mon.a (mon.0) 1938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:11.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:11 smithi067 bash[17385]: cluster 2024-02-10T08:32:10.014402+0000 mgr.y (mgr.14547) 391 : cluster [DBG] pgmap v505: 228 pgs: 32 unknown, 1 active+undersized+remapped, 195 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 32 B/s, 13 objects/s recovering 2024-02-10T08:32:12.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:32:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:32:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:12 smithi134 bash[18722]: audit 2024-02-10T08:32:11.467705+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:12 smithi134 bash[18722]: cluster 2024-02-10T08:32:11.467930+0000 mon.a (mon.0) 1940 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-10T08:32:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:12 smithi134 bash[18722]: audit 2024-02-10T08:32:11.469106+0000 mon.c (mon.2) 651 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:32:12.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:12 smithi134 bash[18722]: audit 2024-02-10T08:32:11.470250+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:32:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[17385]: audit 2024-02-10T08:32:11.467705+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[17385]: cluster 2024-02-10T08:32:11.467930+0000 mon.a (mon.0) 1940 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[17385]: audit 2024-02-10T08:32:11.469106+0000 mon.c (mon.2) 651 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[17385]: audit 2024-02-10T08:32:11.470250+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[24488]: audit 2024-02-10T08:32:11.467705+0000 mon.a (mon.0) 1939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[24488]: cluster 2024-02-10T08:32:11.467930+0000 mon.a (mon.0) 1940 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[24488]: audit 2024-02-10T08:32:11.469106+0000 mon.c (mon.2) 651 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:32:12.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:12 smithi067 bash[24488]: audit 2024-02-10T08:32:11.470250+0000 mon.a (mon.0) 1941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:32:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: cluster 2024-02-10T08:32:12.016031+0000 mgr.y (mgr.14547) 392 : cluster [DBG] pgmap v507: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: audit 2024-02-10T08:32:12.020757+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: audit 2024-02-10T08:32:12.020934+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]': finished 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: cluster 2024-02-10T08:32:12.021027+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: audit 2024-02-10T08:32:12.025451+0000 mon.c (mon.2) 652 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: audit 2024-02-10T08:32:12.031562+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[17385]: audit 2024-02-10T08:32:12.042978+0000 mgr.y (mgr.14547) 393 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: cluster 2024-02-10T08:32:12.016031+0000 mgr.y (mgr.14547) 392 : cluster [DBG] pgmap v507: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: audit 2024-02-10T08:32:12.020757+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: audit 2024-02-10T08:32:12.020934+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]': finished 2024-02-10T08:32:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: cluster 2024-02-10T08:32:12.021027+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-10T08:32:13.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: audit 2024-02-10T08:32:12.025451+0000 mon.c (mon.2) 652 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:32:13.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: audit 2024-02-10T08:32:12.031562+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:32:13.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:13 smithi067 bash[24488]: audit 2024-02-10T08:32:12.042978+0000 mgr.y (mgr.14547) 393 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: cluster 2024-02-10T08:32:12.016031+0000 mgr.y (mgr.14547) 392 : cluster [DBG] pgmap v507: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: audit 2024-02-10T08:32:12.020757+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi067-54245-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: audit 2024-02-10T08:32:12.020934+0000 mon.a (mon.0) 1943 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_count","val": "3"}]': finished 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: cluster 2024-02-10T08:32:12.021027+0000 mon.a (mon.0) 1944 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: audit 2024-02-10T08:32:12.025451+0000 mon.c (mon.2) 652 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: audit 2024-02-10T08:32:12.031562+0000 mon.a (mon.0) 1945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:32:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:13 smithi134 bash[18722]: audit 2024-02-10T08:32:12.042978+0000 mgr.y (mgr.14547) 393 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:13.026176+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]': finished 2024-02-10T08:32:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: cluster 2024-02-10T08:32:13.026369+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-10T08:32:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:13.029060+0000 mon.c (mon.2) 653 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:13.038903+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:14.029054+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:32:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: cluster 2024-02-10T08:32:14.029618+0000 mon.a (mon.0) 1950 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:14.031542+0000 mon.c (mon.2) 654 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:14.031909+0000 mon.c (mon.2) 655 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:14.032105+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[17385]: audit 2024-02-10T08:32:14.032727+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:13.026176+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]': finished 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: cluster 2024-02-10T08:32:13.026369+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:13.029060+0000 mon.c (mon.2) 653 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:13.038903+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:14.029054+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:32:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: cluster 2024-02-10T08:32:14.029618+0000 mon.a (mon.0) 1950 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-10T08:32:14.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:14.031542+0000 mon.c (mon.2) 654 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:32:14.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:14.031909+0000 mon.c (mon.2) 655 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:14.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:14.032105+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:32:14.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:14 smithi067 bash[24488]: audit 2024-02-10T08:32:14.032727+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:14.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:13.026176+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_period","val": "3"}]': finished 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: cluster 2024-02-10T08:32:13.026369+0000 mon.a (mon.0) 1947 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:13.029060+0000 mon.c (mon.2) 653 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:13.038903+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:14.029054+0000 mon.a (mon.0) 1949 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: cluster 2024-02-10T08:32:14.029618+0000 mon.a (mon.0) 1950 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:14.031542+0000 mon.c (mon.2) 654 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:14.031909+0000 mon.c (mon.2) 655 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:14.032105+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:32:14.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:14 smithi134 bash[18722]: audit 2024-02-10T08:32:14.032727+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[17385]: cluster 2024-02-10T08:32:14.016849+0000 mgr.y (mgr.14547) 394 : cluster [DBG] pgmap v510: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[17385]: audit 2024-02-10T08:32:15.032328+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T08:32:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[17385]: audit 2024-02-10T08:32:15.032496+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[17385]: cluster 2024-02-10T08:32:15.032614+0000 mon.a (mon.0) 1955 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[17385]: audit 2024-02-10T08:32:15.037436+0000 mon.c (mon.2) 656 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[17385]: audit 2024-02-10T08:32:15.038856+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[24488]: cluster 2024-02-10T08:32:14.016849+0000 mgr.y (mgr.14547) 394 : cluster [DBG] pgmap v510: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[24488]: audit 2024-02-10T08:32:15.032328+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[24488]: audit 2024-02-10T08:32:15.032496+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[24488]: cluster 2024-02-10T08:32:15.032614+0000 mon.a (mon.0) 1955 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[24488]: audit 2024-02-10T08:32:15.037436+0000 mon.c (mon.2) 656 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:15.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:15 smithi067 bash[24488]: audit 2024-02-10T08:32:15.038856+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:15 smithi134 bash[18722]: cluster 2024-02-10T08:32:14.016849+0000 mgr.y (mgr.14547) 394 : cluster [DBG] pgmap v510: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:15 smithi134 bash[18722]: audit 2024-02-10T08:32:15.032328+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T08:32:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:15 smithi134 bash[18722]: audit 2024-02-10T08:32:15.032496+0000 mon.a (mon.0) 1954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:15 smithi134 bash[18722]: cluster 2024-02-10T08:32:15.032614+0000 mon.a (mon.0) 1955 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-10T08:32:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:15 smithi134 bash[18722]: audit 2024-02-10T08:32:15.037436+0000 mon.c (mon.2) 656 : audit [INF] from='client.? 172.21.15.67:0/4259191004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:15 smithi134 bash[18722]: audit 2024-02-10T08:32:15.038856+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]: dispatch 2024-02-10T08:32:17.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: cluster 2024-02-10T08:32:16.017723+0000 mgr.y (mgr.14547) 395 : cluster [DBG] pgmap v513: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.036687+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: cluster 2024-02-10T08:32:16.036870+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.061154+0000 mon.c (mon.2) 657 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.061724+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.062952+0000 mon.c (mon.2) 658 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.063389+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.064088+0000 mon.c (mon.2) 659 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17385]: audit 2024-02-10T08:32:16.064596+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:17.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: cluster 2024-02-10T08:32:16.017723+0000 mgr.y (mgr.14547) 395 : cluster [DBG] pgmap v513: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.036687+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: cluster 2024-02-10T08:32:16.036870+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.061154+0000 mon.c (mon.2) 657 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.061724+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.062952+0000 mon.c (mon.2) 658 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.063389+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.064088+0000 mon.c (mon.2) 659 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:17.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[24488]: audit 2024-02-10T08:32:16.064596+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: cluster 2024-02-10T08:32:16.017723+0000 mgr.y (mgr.14547) 395 : cluster [DBG] pgmap v513: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.036687+0000 mon.a (mon.0) 1957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi067-54245-56"}]': finished 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: cluster 2024-02-10T08:32:16.036870+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.061154+0000 mon.c (mon.2) 657 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.061724+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.062952+0000 mon.c (mon.2) 658 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.063389+0000 mon.a (mon.0) 1960 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.064088+0000 mon.c (mon.2) 659 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:17.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:17 smithi134 bash[18722]: audit 2024-02-10T08:32:16.064596+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[17385]: audit 2024-02-10T08:32:17.044203+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[17385]: cluster 2024-02-10T08:32:17.044322+0000 mon.a (mon.0) 1963 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-10T08:32:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[17385]: audit 2024-02-10T08:32:17.046033+0000 mon.c (mon.2) 660 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[17385]: audit 2024-02-10T08:32:17.046693+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:18.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:32:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:17] "GET /metrics HTTP/1.1" 200 187744 "" "Prometheus/2.33.4" 2024-02-10T08:32:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[24488]: audit 2024-02-10T08:32:17.044203+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[24488]: cluster 2024-02-10T08:32:17.044322+0000 mon.a (mon.0) 1963 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-10T08:32:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[24488]: audit 2024-02-10T08:32:17.046033+0000 mon.c (mon.2) 660 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:18 smithi067 bash[24488]: audit 2024-02-10T08:32:17.046693+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:18.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:18 smithi134 bash[18722]: audit 2024-02-10T08:32:17.044203+0000 mon.a (mon.0) 1962 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi067-54245-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:18 smithi134 bash[18722]: cluster 2024-02-10T08:32:17.044322+0000 mon.a (mon.0) 1963 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-10T08:32:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:18 smithi134 bash[18722]: audit 2024-02-10T08:32:17.046033+0000 mon.c (mon.2) 660 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:18 smithi134 bash[18722]: audit 2024-02-10T08:32:17.046693+0000 mon.a (mon.0) 1964 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:19.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:19 smithi067 bash[17385]: cluster 2024-02-10T08:32:18.018514+0000 mgr.y (mgr.14547) 396 : cluster [DBG] pgmap v516: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:32:19.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:19 smithi067 bash[17385]: cluster 2024-02-10T08:32:18.513533+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-10T08:32:19.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:19 smithi067 bash[24488]: cluster 2024-02-10T08:32:18.018514+0000 mgr.y (mgr.14547) 396 : cluster [DBG] pgmap v516: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:32:19.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:19 smithi067 bash[24488]: cluster 2024-02-10T08:32:18.513533+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-10T08:32:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:19 smithi134 bash[18722]: cluster 2024-02-10T08:32:18.018514+0000 mgr.y (mgr.14547) 396 : cluster [DBG] pgmap v516: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:32:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:19 smithi134 bash[18722]: cluster 2024-02-10T08:32:18.513533+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-10T08:32:20.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:20 smithi067 bash[24488]: audit 2024-02-10T08:32:19.066629+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:20.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:20 smithi067 bash[24488]: cluster 2024-02-10T08:32:19.066798+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-10T08:32:20.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:20 smithi067 bash[17385]: audit 2024-02-10T08:32:19.066629+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:20.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:20 smithi067 bash[17385]: cluster 2024-02-10T08:32:19.066798+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-10T08:32:20.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:20 smithi134 bash[18722]: audit 2024-02-10T08:32:19.066629+0000 mon.a (mon.0) 1966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi067-54245-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:20.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:20 smithi134 bash[18722]: cluster 2024-02-10T08:32:19.066798+0000 mon.a (mon.0) 1967 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-10T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:21 smithi067 bash[17385]: cluster 2024-02-10T08:32:20.019447+0000 mgr.y (mgr.14547) 397 : cluster [DBG] pgmap v519: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:32:21.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:21 smithi067 bash[17385]: cluster 2024-02-10T08:32:20.071245+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-10T08:32:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:21 smithi067 bash[24488]: cluster 2024-02-10T08:32:20.019447+0000 mgr.y (mgr.14547) 397 : cluster [DBG] pgmap v519: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:32:21.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:21 smithi067 bash[24488]: cluster 2024-02-10T08:32:20.071245+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-10T08:32:21.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:21 smithi134 bash[18722]: cluster 2024-02-10T08:32:20.019447+0000 mgr.y (mgr.14547) 397 : cluster [DBG] pgmap v519: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:32:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:21 smithi134 bash[18722]: cluster 2024-02-10T08:32:20.071245+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-10T08:32:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:22 smithi067 bash[17385]: cluster 2024-02-10T08:32:21.086125+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-10T08:32:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:22 smithi067 bash[17385]: audit 2024-02-10T08:32:21.092892+0000 mon.c (mon.2) 661 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:22 smithi067 bash[17385]: audit 2024-02-10T08:32:21.098601+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:22 smithi067 bash[24488]: cluster 2024-02-10T08:32:21.086125+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-10T08:32:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:22 smithi067 bash[24488]: audit 2024-02-10T08:32:21.092892+0000 mon.c (mon.2) 661 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:22.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:22 smithi067 bash[24488]: audit 2024-02-10T08:32:21.098601+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:22.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:32:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:32:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:22 smithi134 bash[18722]: cluster 2024-02-10T08:32:21.086125+0000 mon.a (mon.0) 1969 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-10T08:32:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:22 smithi134 bash[18722]: audit 2024-02-10T08:32:21.092892+0000 mon.c (mon.2) 661 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:22 smithi134 bash[18722]: audit 2024-02-10T08:32:21.098601+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[18722]: cluster 2024-02-10T08:32:22.021512+0000 mgr.y (mgr.14547) 398 : cluster [DBG] pgmap v522: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[18722]: audit 2024-02-10T08:32:22.053994+0000 mgr.y (mgr.14547) 399 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[18722]: audit 2024-02-10T08:32:22.100368+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[18722]: cluster 2024-02-10T08:32:22.100492+0000 mon.a (mon.0) 1972 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[18722]: audit 2024-02-10T08:32:22.106650+0000 mon.c (mon.2) 662 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:23 smithi134 bash[18722]: audit 2024-02-10T08:32:22.112748+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[17385]: cluster 2024-02-10T08:32:22.021512+0000 mgr.y (mgr.14547) 398 : cluster [DBG] pgmap v522: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[17385]: audit 2024-02-10T08:32:22.053994+0000 mgr.y (mgr.14547) 399 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[17385]: audit 2024-02-10T08:32:22.100368+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[17385]: cluster 2024-02-10T08:32:22.100492+0000 mon.a (mon.0) 1972 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[17385]: audit 2024-02-10T08:32:22.106650+0000 mon.c (mon.2) 662 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[17385]: audit 2024-02-10T08:32:22.112748+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[24488]: cluster 2024-02-10T08:32:22.021512+0000 mgr.y (mgr.14547) 398 : cluster [DBG] pgmap v522: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[24488]: audit 2024-02-10T08:32:22.053994+0000 mgr.y (mgr.14547) 399 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[24488]: audit 2024-02-10T08:32:22.100368+0000 mon.a (mon.0) 1971 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[24488]: cluster 2024-02-10T08:32:22.100492+0000 mon.a (mon.0) 1972 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-10T08:32:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[24488]: audit 2024-02-10T08:32:22.106650+0000 mon.c (mon.2) 662 : audit [INF] from='client.? 172.21.15.67:0/52080711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:23.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:23 smithi067 bash[24488]: audit 2024-02-10T08:32:22.112748+0000 mon.a (mon.0) 1973 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]: dispatch 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.114751+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: cluster 2024-02-10T08:32:23.114877+0000 mon.a (mon.0) 1975 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.140992+0000 mon.c (mon.2) 663 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.141600+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.142921+0000 mon.c (mon.2) 664 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.143401+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.144079+0000 mon.c (mon.2) 665 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:24 smithi134 bash[18722]: audit 2024-02-10T08:32:23.144506+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.114751+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: cluster 2024-02-10T08:32:23.114877+0000 mon.a (mon.0) 1975 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.140992+0000 mon.c (mon.2) 663 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.141600+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.142921+0000 mon.c (mon.2) 664 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.143401+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.144079+0000 mon.c (mon.2) 665 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[17385]: audit 2024-02-10T08:32:23.144506+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.114751+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi067-54245-57"}]': finished 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: cluster 2024-02-10T08:32:23.114877+0000 mon.a (mon.0) 1975 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.140992+0000 mon.c (mon.2) 663 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.141600+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.142921+0000 mon.c (mon.2) 664 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.143401+0000 mon.a (mon.0) 1977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.144079+0000 mon.c (mon.2) 665 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:24.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:24 smithi067 bash[24488]: audit 2024-02-10T08:32:23.144506+0000 mon.a (mon.0) 1978 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:25 smithi134 bash[18722]: cluster 2024-02-10T08:32:24.022068+0000 mgr.y (mgr.14547) 400 : cluster [DBG] pgmap v525: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T08:32:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:25 smithi134 bash[18722]: audit 2024-02-10T08:32:24.139107+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:25 smithi134 bash[18722]: cluster 2024-02-10T08:32:24.139172+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-10T08:32:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:25 smithi134 bash[18722]: audit 2024-02-10T08:32:24.144799+0000 mon.c (mon.2) 666 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:25 smithi134 bash[18722]: audit 2024-02-10T08:32:24.146655+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[17385]: cluster 2024-02-10T08:32:24.022068+0000 mgr.y (mgr.14547) 400 : cluster [DBG] pgmap v525: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T08:32:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[17385]: audit 2024-02-10T08:32:24.139107+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[17385]: cluster 2024-02-10T08:32:24.139172+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[17385]: audit 2024-02-10T08:32:24.144799+0000 mon.c (mon.2) 666 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[17385]: audit 2024-02-10T08:32:24.146655+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[24488]: cluster 2024-02-10T08:32:24.022068+0000 mgr.y (mgr.14547) 400 : cluster [DBG] pgmap v525: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[24488]: audit 2024-02-10T08:32:24.139107+0000 mon.a (mon.0) 1979 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[24488]: cluster 2024-02-10T08:32:24.139172+0000 mon.a (mon.0) 1980 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[24488]: audit 2024-02-10T08:32:24.144799+0000 mon.c (mon.2) 666 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:25 smithi067 bash[24488]: audit 2024-02-10T08:32:24.146655+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:26 smithi134 bash[18722]: cluster 2024-02-10T08:32:25.600059+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-10T08:32:26.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:26 smithi067 bash[17385]: cluster 2024-02-10T08:32:25.600059+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-10T08:32:26.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:26 smithi067 bash[24488]: cluster 2024-02-10T08:32:25.600059+0000 mon.a (mon.0) 1982 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: cluster 2024-02-10T08:32:26.022764+0000 mgr.y (mgr.14547) 401 : cluster [DBG] pgmap v528: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: audit 2024-02-10T08:32:26.160826+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: cluster 2024-02-10T08:32:26.160968+0000 mon.a (mon.0) 1984 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: audit 2024-02-10T08:32:27.118841+0000 mon.c (mon.2) 667 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: audit 2024-02-10T08:32:27.119480+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: audit 2024-02-10T08:32:27.120641+0000 mon.c (mon.2) 668 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]: dispatch 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: audit 2024-02-10T08:32:27.121279+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]: dispatch 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: audit 2024-02-10T08:32:27.161102+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]': finished 2024-02-10T08:32:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:27 smithi134 bash[18722]: cluster 2024-02-10T08:32:27.161307+0000 mon.a (mon.0) 1988 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-10T08:32:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: cluster 2024-02-10T08:32:26.022764+0000 mgr.y (mgr.14547) 401 : cluster [DBG] pgmap v528: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: audit 2024-02-10T08:32:26.160826+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: cluster 2024-02-10T08:32:26.160968+0000 mon.a (mon.0) 1984 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: audit 2024-02-10T08:32:27.118841+0000 mon.c (mon.2) 667 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: audit 2024-02-10T08:32:27.119480+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: audit 2024-02-10T08:32:27.120641+0000 mon.c (mon.2) 668 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]: dispatch 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: audit 2024-02-10T08:32:27.121279+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]: dispatch 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: audit 2024-02-10T08:32:27.161102+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]': finished 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17385]: cluster 2024-02-10T08:32:27.161307+0000 mon.a (mon.0) 1988 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: cluster 2024-02-10T08:32:26.022764+0000 mgr.y (mgr.14547) 401 : cluster [DBG] pgmap v528: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: audit 2024-02-10T08:32:26.160826+0000 mon.a (mon.0) 1983 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi067-54245-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: cluster 2024-02-10T08:32:26.160968+0000 mon.a (mon.0) 1984 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-10T08:32:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: audit 2024-02-10T08:32:27.118841+0000 mon.c (mon.2) 667 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: audit 2024-02-10T08:32:27.119480+0000 mon.a (mon.0) 1985 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: audit 2024-02-10T08:32:27.120641+0000 mon.c (mon.2) 668 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]: dispatch 2024-02-10T08:32:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: audit 2024-02-10T08:32:27.121279+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]: dispatch 2024-02-10T08:32:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: audit 2024-02-10T08:32:27.161102+0000 mon.a (mon.0) 1987 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-43"}]': finished 2024-02-10T08:32:27.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[24488]: cluster 2024-02-10T08:32:27.161307+0000 mon.a (mon.0) 1988 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-10T08:32:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:32:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:27] "GET /metrics HTTP/1.1" 200 187744 "" "Prometheus/2.33.4" 2024-02-10T08:32:29.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:29 smithi134 bash[18722]: cluster 2024-02-10T08:32:28.023589+0000 mgr.y (mgr.14547) 402 : cluster [DBG] pgmap v531: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:32:29.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:29 smithi134 bash[18722]: cluster 2024-02-10T08:32:28.164983+0000 mon.a (mon.0) 1989 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-10T08:32:29.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:29 smithi134 bash[18722]: audit 2024-02-10T08:32:28.168516+0000 mon.c (mon.2) 669 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:29.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:29 smithi134 bash[18722]: audit 2024-02-10T08:32:28.640798+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[24488]: cluster 2024-02-10T08:32:28.023589+0000 mgr.y (mgr.14547) 402 : cluster [DBG] pgmap v531: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:32:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[24488]: cluster 2024-02-10T08:32:28.164983+0000 mon.a (mon.0) 1989 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-10T08:32:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[24488]: audit 2024-02-10T08:32:28.168516+0000 mon.c (mon.2) 669 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[24488]: audit 2024-02-10T08:32:28.640798+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[17385]: cluster 2024-02-10T08:32:28.023589+0000 mgr.y (mgr.14547) 402 : cluster [DBG] pgmap v531: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:32:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[17385]: cluster 2024-02-10T08:32:28.164983+0000 mon.a (mon.0) 1989 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-10T08:32:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[17385]: audit 2024-02-10T08:32:28.168516+0000 mon.c (mon.2) 669 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:29 smithi067 bash[17385]: audit 2024-02-10T08:32:28.640798+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:30 smithi134 bash[18722]: audit 2024-02-10T08:32:29.173741+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:30 smithi134 bash[18722]: cluster 2024-02-10T08:32:29.173860+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-10T08:32:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:30 smithi134 bash[18722]: audit 2024-02-10T08:32:29.175807+0000 mon.c (mon.2) 670 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:30 smithi134 bash[18722]: audit 2024-02-10T08:32:29.176743+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:30 smithi134 bash[18722]: audit 2024-02-10T08:32:29.179697+0000 mon.c (mon.2) 671 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:30.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:30 smithi134 bash[18722]: audit 2024-02-10T08:32:29.181373+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[17385]: audit 2024-02-10T08:32:29.173741+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[17385]: cluster 2024-02-10T08:32:29.173860+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-10T08:32:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[17385]: audit 2024-02-10T08:32:29.175807+0000 mon.c (mon.2) 670 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[17385]: audit 2024-02-10T08:32:29.176743+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[17385]: audit 2024-02-10T08:32:29.179697+0000 mon.c (mon.2) 671 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[17385]: audit 2024-02-10T08:32:29.181373+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[24488]: audit 2024-02-10T08:32:29.173741+0000 mon.a (mon.0) 1991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[24488]: cluster 2024-02-10T08:32:29.173860+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[24488]: audit 2024-02-10T08:32:29.175807+0000 mon.c (mon.2) 670 : audit [INF] from='client.? 172.21.15.67:0/4109817350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[24488]: audit 2024-02-10T08:32:29.176743+0000 mon.a (mon.0) 1993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]: dispatch 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[24488]: audit 2024-02-10T08:32:29.179697+0000 mon.c (mon.2) 671 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:30 smithi067 bash[24488]: audit 2024-02-10T08:32:29.181373+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:31 smithi134 bash[18722]: cluster 2024-02-10T08:32:30.024413+0000 mgr.y (mgr.14547) 403 : cluster [DBG] pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:31 smithi134 bash[18722]: audit 2024-02-10T08:32:30.187827+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:31 smithi134 bash[18722]: audit 2024-02-10T08:32:30.188028+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:31 smithi134 bash[18722]: cluster 2024-02-10T08:32:30.188274+0000 mon.a (mon.0) 1997 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-10T08:32:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:31 smithi134 bash[18722]: audit 2024-02-10T08:32:30.253104+0000 mon.c (mon.2) 672 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:31 smithi134 bash[18722]: audit 2024-02-10T08:32:30.686547+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[17385]: cluster 2024-02-10T08:32:30.024413+0000 mgr.y (mgr.14547) 403 : cluster [DBG] pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[17385]: audit 2024-02-10T08:32:30.187827+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[17385]: audit 2024-02-10T08:32:30.188028+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[17385]: cluster 2024-02-10T08:32:30.188274+0000 mon.a (mon.0) 1997 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-10T08:32:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[17385]: audit 2024-02-10T08:32:30.253104+0000 mon.c (mon.2) 672 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[17385]: audit 2024-02-10T08:32:30.686547+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[24488]: cluster 2024-02-10T08:32:30.024413+0000 mgr.y (mgr.14547) 403 : cluster [DBG] pgmap v534: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[24488]: audit 2024-02-10T08:32:30.187827+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi067-54245-58"}]': finished 2024-02-10T08:32:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[24488]: audit 2024-02-10T08:32:30.188028+0000 mon.a (mon.0) 1996 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[24488]: cluster 2024-02-10T08:32:30.188274+0000 mon.a (mon.0) 1997 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-10T08:32:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[24488]: audit 2024-02-10T08:32:30.253104+0000 mon.c (mon.2) 672 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:31.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:31 smithi067 bash[24488]: audit 2024-02-10T08:32:30.686547+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:32.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:32:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[18722]: audit 2024-02-10T08:32:31.189538+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[18722]: cluster 2024-02-10T08:32:31.189648+0000 mon.a (mon.0) 2000 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-10T08:32:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[18722]: audit 2024-02-10T08:32:31.194639+0000 mon.c (mon.2) 673 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[18722]: audit 2024-02-10T08:32:31.195545+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.67:0/2089063555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[18722]: audit 2024-02-10T08:32:31.203074+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:32 smithi134 bash[18722]: audit 2024-02-10T08:32:31.207829+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[17385]: audit 2024-02-10T08:32:31.189538+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[17385]: cluster 2024-02-10T08:32:31.189648+0000 mon.a (mon.0) 2000 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-10T08:32:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[17385]: audit 2024-02-10T08:32:31.194639+0000 mon.c (mon.2) 673 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[17385]: audit 2024-02-10T08:32:31.195545+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.67:0/2089063555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[17385]: audit 2024-02-10T08:32:31.203074+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[17385]: audit 2024-02-10T08:32:31.207829+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[24488]: audit 2024-02-10T08:32:31.189538+0000 mon.a (mon.0) 1999 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[24488]: cluster 2024-02-10T08:32:31.189648+0000 mon.a (mon.0) 2000 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[24488]: audit 2024-02-10T08:32:31.194639+0000 mon.c (mon.2) 673 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[24488]: audit 2024-02-10T08:32:31.195545+0000 mon.b (mon.1) 79 : audit [INF] from='client.? 172.21.15.67:0/2089063555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[24488]: audit 2024-02-10T08:32:31.203074+0000 mon.a (mon.0) 2001 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:32 smithi067 bash[24488]: audit 2024-02-10T08:32:31.207829+0000 mon.a (mon.0) 2002 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: cluster 2024-02-10T08:32:32.026626+0000 mgr.y (mgr.14547) 404 : cluster [DBG] pgmap v537: 260 pgs: 32 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: audit 2024-02-10T08:32:32.059120+0000 mgr.y (mgr.14547) 405 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: audit 2024-02-10T08:32:32.194090+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: audit 2024-02-10T08:32:32.194328+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]': finished 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: cluster 2024-02-10T08:32:32.194503+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: audit 2024-02-10T08:32:32.199554+0000 mon.c (mon.2) 674 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]: dispatch 2024-02-10T08:32:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:33 smithi134 bash[18722]: audit 2024-02-10T08:32:32.213355+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]: dispatch 2024-02-10T08:32:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: cluster 2024-02-10T08:32:32.026626+0000 mgr.y (mgr.14547) 404 : cluster [DBG] pgmap v537: 260 pgs: 32 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:32:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: audit 2024-02-10T08:32:32.059120+0000 mgr.y (mgr.14547) 405 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: audit 2024-02-10T08:32:32.194090+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: audit 2024-02-10T08:32:32.194328+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]': finished 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: cluster 2024-02-10T08:32:32.194503+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: audit 2024-02-10T08:32:32.199554+0000 mon.c (mon.2) 674 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]: dispatch 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[17385]: audit 2024-02-10T08:32:32.213355+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]: dispatch 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: cluster 2024-02-10T08:32:32.026626+0000 mgr.y (mgr.14547) 404 : cluster [DBG] pgmap v537: 260 pgs: 32 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: audit 2024-02-10T08:32:32.059120+0000 mgr.y (mgr.14547) 405 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: audit 2024-02-10T08:32:32.194090+0000 mon.a (mon.0) 2003 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi067-54245-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: audit 2024-02-10T08:32:32.194328+0000 mon.a (mon.0) 2004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-45"}]': finished 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: cluster 2024-02-10T08:32:32.194503+0000 mon.a (mon.0) 2005 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-10T08:32:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: audit 2024-02-10T08:32:32.199554+0000 mon.c (mon.2) 674 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]: dispatch 2024-02-10T08:32:33.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:33 smithi067 bash[24488]: audit 2024-02-10T08:32:32.213355+0000 mon.a (mon.0) 2006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]: dispatch 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: cluster 2024-02-10T08:32:33.201250+0000 mon.a (mon.0) 2007 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.208865+0000 mon.a (mon.0) 2008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]': finished 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: cluster 2024-02-10T08:32:33.209097+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.223171+0000 mon.c (mon.2) 675 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.229591+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.237629+0000 mon.c (mon.2) 676 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.238235+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.239080+0000 mon.c (mon.2) 677 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.239404+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.239943+0000 mon.c (mon.2) 678 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:34.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:34 smithi134 bash[18722]: audit 2024-02-10T08:32:33.240289+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: cluster 2024-02-10T08:32:33.201250+0000 mon.a (mon.0) 2007 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.208865+0000 mon.a (mon.0) 2008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]': finished 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: cluster 2024-02-10T08:32:33.209097+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.223171+0000 mon.c (mon.2) 675 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.229591+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.237629+0000 mon.c (mon.2) 676 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.238235+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.239080+0000 mon.c (mon.2) 677 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.239404+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.239943+0000 mon.c (mon.2) 678 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[17385]: audit 2024-02-10T08:32:33.240289+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: cluster 2024-02-10T08:32:33.201250+0000 mon.a (mon.0) 2007 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:32:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.208865+0000 mon.a (mon.0) 2008 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-45", "mode": "writeback"}]': finished 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: cluster 2024-02-10T08:32:33.209097+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.223171+0000 mon.c (mon.2) 675 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.229591+0000 mon.a (mon.0) 2010 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.237629+0000 mon.c (mon.2) 676 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.238235+0000 mon.a (mon.0) 2011 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.239080+0000 mon.c (mon.2) 677 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.239404+0000 mon.a (mon.0) 2012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.239943+0000 mon.c (mon.2) 678 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:34.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:34 smithi067 bash[24488]: audit 2024-02-10T08:32:33.240289+0000 mon.a (mon.0) 2013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: cluster 2024-02-10T08:32:34.027745+0000 mgr.y (mgr.14547) 406 : cluster [DBG] pgmap v540: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:34.232938+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:34.233077+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: cluster 2024-02-10T08:32:34.233402+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:34.234520+0000 mon.c (mon.2) 679 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:34.235127+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:34.235446+0000 mon.c (mon.2) 680 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:34.243582+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:35 smithi134 bash[18722]: audit 2024-02-10T08:32:35.235905+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:32:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: cluster 2024-02-10T08:32:34.027745+0000 mgr.y (mgr.14547) 406 : cluster [DBG] pgmap v540: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:34.232938+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:34.233077+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: cluster 2024-02-10T08:32:34.233402+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:34.234520+0000 mon.c (mon.2) 679 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:34.235127+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:34.235446+0000 mon.c (mon.2) 680 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:34.243582+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[17385]: audit 2024-02-10T08:32:35.235905+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:32:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: cluster 2024-02-10T08:32:34.027745+0000 mgr.y (mgr.14547) 406 : cluster [DBG] pgmap v540: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:34.232938+0000 mon.a (mon.0) 2014 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:34.233077+0000 mon.a (mon.0) 2015 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi067-54245-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: cluster 2024-02-10T08:32:34.233402+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:34.234520+0000 mon.c (mon.2) 679 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:34.235127+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:34.235446+0000 mon.c (mon.2) 680 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:34.243582+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:36.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:35 smithi067 bash[24488]: audit 2024-02-10T08:32:35.235905+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: audit 2024-02-10T08:32:35.237571+0000 mon.c (mon.2) 681 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: cluster 2024-02-10T08:32:35.685846+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: audit 2024-02-10T08:32:35.687419+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: cluster 2024-02-10T08:32:36.230513+0000 mon.a (mon.0) 2022 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: audit 2024-02-10T08:32:36.241528+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: audit 2024-02-10T08:32:36.241728+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: cluster 2024-02-10T08:32:36.241866+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: audit 2024-02-10T08:32:36.249847+0000 mon.c (mon.2) 682 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:32:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:36 smithi134 bash[18722]: audit 2024-02-10T08:32:36.251141+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:32:37.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: audit 2024-02-10T08:32:35.237571+0000 mon.c (mon.2) 681 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: cluster 2024-02-10T08:32:35.685846+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: audit 2024-02-10T08:32:35.687419+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: cluster 2024-02-10T08:32:36.230513+0000 mon.a (mon.0) 2022 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: audit 2024-02-10T08:32:36.241528+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: audit 2024-02-10T08:32:36.241728+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: cluster 2024-02-10T08:32:36.241866+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: audit 2024-02-10T08:32:36.249847+0000 mon.c (mon.2) 682 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[24488]: audit 2024-02-10T08:32:36.251141+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: audit 2024-02-10T08:32:35.237571+0000 mon.c (mon.2) 681 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:37.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: cluster 2024-02-10T08:32:35.685846+0000 mon.a (mon.0) 2020 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: audit 2024-02-10T08:32:35.687419+0000 mon.a (mon.0) 2021 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: cluster 2024-02-10T08:32:36.230513+0000 mon.a (mon.0) 2022 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: audit 2024-02-10T08:32:36.241528+0000 mon.a (mon.0) 2023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi067-54245-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: audit 2024-02-10T08:32:36.241728+0000 mon.a (mon.0) 2024 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: cluster 2024-02-10T08:32:36.241866+0000 mon.a (mon.0) 2025 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: audit 2024-02-10T08:32:36.249847+0000 mon.c (mon.2) 682 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:32:37.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:36 smithi067 bash[17385]: audit 2024-02-10T08:32:36.251141+0000 mon.a (mon.0) 2026 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:32:38.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:37 smithi134 bash[18722]: cluster 2024-02-10T08:32:36.028496+0000 mgr.y (mgr.14547) 407 : cluster [DBG] pgmap v543: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:37 smithi134 bash[18722]: audit 2024-02-10T08:32:37.243734+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:32:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:37 smithi134 bash[18722]: cluster 2024-02-10T08:32:37.243911+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-10T08:32:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:37 smithi134 bash[18722]: audit 2024-02-10T08:32:37.249621+0000 mon.c (mon.2) 683 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:32:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:37 smithi134 bash[18722]: audit 2024-02-10T08:32:37.250247+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:32:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:37] "GET /metrics HTTP/1.1" 200 187875 "" "Prometheus/2.33.4" 2024-02-10T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[24488]: cluster 2024-02-10T08:32:36.028496+0000 mgr.y (mgr.14547) 407 : cluster [DBG] pgmap v543: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[24488]: audit 2024-02-10T08:32:37.243734+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:32:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[24488]: cluster 2024-02-10T08:32:37.243911+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[24488]: audit 2024-02-10T08:32:37.249621+0000 mon.c (mon.2) 683 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[24488]: audit 2024-02-10T08:32:37.250247+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[17385]: cluster 2024-02-10T08:32:36.028496+0000 mgr.y (mgr.14547) 407 : cluster [DBG] pgmap v543: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[17385]: audit 2024-02-10T08:32:37.243734+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[17385]: cluster 2024-02-10T08:32:37.243911+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[17385]: audit 2024-02-10T08:32:37.249621+0000 mon.c (mon.2) 683 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:32:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:37 smithi067 bash[17385]: audit 2024-02-10T08:32:37.250247+0000 mon.a (mon.0) 2029 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:32:39.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: cluster 2024-02-10T08:32:38.029356+0000 mgr.y (mgr.14547) 408 : cluster [DBG] pgmap v546: 236 pgs: 6 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: audit 2024-02-10T08:32:38.244302+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T08:32:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: cluster 2024-02-10T08:32:38.244719+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-10T08:32:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: audit 2024-02-10T08:32:38.247349+0000 mon.c (mon.2) 684 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:32:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: audit 2024-02-10T08:32:38.247847+0000 mon.c (mon.2) 685 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: audit 2024-02-10T08:32:38.248322+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:32:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:39 smithi134 bash[18722]: audit 2024-02-10T08:32:38.249267+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: cluster 2024-02-10T08:32:38.029356+0000 mgr.y (mgr.14547) 408 : cluster [DBG] pgmap v546: 236 pgs: 6 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: audit 2024-02-10T08:32:38.244302+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T08:32:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: cluster 2024-02-10T08:32:38.244719+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-10T08:32:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: audit 2024-02-10T08:32:38.247349+0000 mon.c (mon.2) 684 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:32:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: audit 2024-02-10T08:32:38.247847+0000 mon.c (mon.2) 685 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: audit 2024-02-10T08:32:38.248322+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[24488]: audit 2024-02-10T08:32:38.249267+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: cluster 2024-02-10T08:32:38.029356+0000 mgr.y (mgr.14547) 408 : cluster [DBG] pgmap v546: 236 pgs: 6 creating+peering, 2 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: audit 2024-02-10T08:32:38.244302+0000 mon.a (mon.0) 2030 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: cluster 2024-02-10T08:32:38.244719+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: audit 2024-02-10T08:32:38.247349+0000 mon.c (mon.2) 684 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: audit 2024-02-10T08:32:38.247847+0000 mon.c (mon.2) 685 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: audit 2024-02-10T08:32:38.248322+0000 mon.a (mon.0) 2032 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:32:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:39 smithi067 bash[17385]: audit 2024-02-10T08:32:38.249267+0000 mon.a (mon.0) 2033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.277 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:bo.RoundTripAppendPP (3049 ms) 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3017 ms) 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2915 ms) 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3017 ms) 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3022 ms) 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio (126003 ms total) 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-10T08:32:40.278 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3480 ms) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (8587 ms) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (15121 ms) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (2795 ms) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29983 ms total) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3007 ms) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3007 ms total) 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: 2024-02-10T08:32:40.279 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14103 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6925 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7043 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7038 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3009 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (6898 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7042 ms) 2024-02-10T08:32:40.280 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7062 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7068 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7052 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7037 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7079 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7076 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3018 ms) 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-10T08:32:40.281 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7064 ms) 2024-02-10T08:32:40.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: audit 2024-02-10T08:32:39.247736+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T08:32:40.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: audit 2024-02-10T08:32:39.247924+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: cluster 2024-02-10T08:32:39.248049+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-10T08:32:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: audit 2024-02-10T08:32:39.250502+0000 mon.c (mon.2) 686 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: audit 2024-02-10T08:32:39.256880+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: audit 2024-02-10T08:32:39.310298+0000 mon.c (mon.2) 687 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:40 smithi134 bash[18722]: audit 2024-02-10T08:32:39.311076+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: audit 2024-02-10T08:32:39.247736+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T08:32:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: audit 2024-02-10T08:32:39.247924+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: cluster 2024-02-10T08:32:39.248049+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-10T08:32:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: audit 2024-02-10T08:32:39.250502+0000 mon.c (mon.2) 686 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: audit 2024-02-10T08:32:39.256880+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: audit 2024-02-10T08:32:39.310298+0000 mon.c (mon.2) 687 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[17385]: audit 2024-02-10T08:32:39.311076+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: audit 2024-02-10T08:32:39.247736+0000 mon.a (mon.0) 2034 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: audit 2024-02-10T08:32:39.247924+0000 mon.a (mon.0) 2035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: cluster 2024-02-10T08:32:39.248049+0000 mon.a (mon.0) 2036 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: audit 2024-02-10T08:32:39.250502+0000 mon.c (mon.2) 686 : audit [INF] from='client.? 172.21.15.67:0/1145876369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: audit 2024-02-10T08:32:39.256880+0000 mon.a (mon.0) 2037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]: dispatch 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: audit 2024-02-10T08:32:39.310298+0000 mon.c (mon.2) 687 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:40.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:40 smithi067 bash[24488]: audit 2024-02-10T08:32:39.311076+0000 mon.a (mon.0) 2038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: cluster 2024-02-10T08:32:40.030228+0000 mgr.y (mgr.14547) 409 : cluster [DBG] pgmap v549: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.270910+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.271016+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:32:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: cluster 2024-02-10T08:32:40.271095+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.273259+0000 mon.c (mon.2) 688 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.274974+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.297586+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.299290+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:40.299900+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[24488]: audit 2024-02-10T08:32:41.073729+0000 mon.c (mon.2) 689 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: cluster 2024-02-10T08:32:40.030228+0000 mgr.y (mgr.14547) 409 : cluster [DBG] pgmap v549: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.270910+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.271016+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: cluster 2024-02-10T08:32:40.271095+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.273259+0000 mon.c (mon.2) 688 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.274974+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:41.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.297586+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:41.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.299290+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:41.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:40.299900+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:41.622 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:41 smithi067 bash[17385]: audit 2024-02-10T08:32:41.073729+0000 mon.c (mon.2) 689 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:32:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: cluster 2024-02-10T08:32:40.030228+0000 mgr.y (mgr.14547) 409 : cluster [DBG] pgmap v549: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.270910+0000 mon.a (mon.0) 2039 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi067-54245-60"}]': finished 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.271016+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: cluster 2024-02-10T08:32:40.271095+0000 mon.a (mon.0) 2041 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.273259+0000 mon.c (mon.2) 688 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.274974+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.297586+0000 mon.a (mon.0) 2043 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.299290+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:40.299900+0000 mon.a (mon.0) 2045 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:41 smithi134 bash[18722]: audit 2024-02-10T08:32:41.073729+0000 mon.c (mon.2) 689 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:32:42.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:32:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:32:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.275429+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]': finished 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.275727+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: cluster 2024-02-10T08:32:41.275918+0000 mon.a (mon.0) 2048 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.277539+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi067-54245-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.318475+0000 mon.c (mon.2) 690 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.318997+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.319731+0000 mon.c (mon.2) 691 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.320144+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.380505+0000 mon.c (mon.2) 692 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:32:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.381848+0000 mon.c (mon.2) 693 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:32:43.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:43 smithi134 bash[18722]: audit 2024-02-10T08:32:41.389575+0000 mon.a (mon.0) 2052 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:32:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.275429+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]': finished 2024-02-10T08:32:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.275727+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: cluster 2024-02-10T08:32:41.275918+0000 mon.a (mon.0) 2048 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.277539+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi067-54245-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.318475+0000 mon.c (mon.2) 690 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.318997+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.319731+0000 mon.c (mon.2) 691 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.320144+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.380505+0000 mon.c (mon.2) 692 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.381848+0000 mon.c (mon.2) 693 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[17385]: audit 2024-02-10T08:32:41.389575+0000 mon.a (mon.0) 2052 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.275429+0000 mon.a (mon.0) 2046 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]': finished 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.275727+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: cluster 2024-02-10T08:32:41.275918+0000 mon.a (mon.0) 2048 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.277539+0000 mon.a (mon.0) 2049 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi067-54245-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.318475+0000 mon.c (mon.2) 690 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.318997+0000 mon.a (mon.0) 2050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:43.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.319731+0000 mon.c (mon.2) 691 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:43.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.320144+0000 mon.a (mon.0) 2051 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-45"}]: dispatch 2024-02-10T08:32:43.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.380505+0000 mon.c (mon.2) 692 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:32:43.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.381848+0000 mon.c (mon.2) 693 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:32:43.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:43 smithi067 bash[24488]: audit 2024-02-10T08:32:41.389575+0000 mon.a (mon.0) 2052 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: cluster 2024-02-10T08:32:42.032459+0000 mgr.y (mgr.14547) 410 : cluster [DBG] pgmap v552: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: audit 2024-02-10T08:32:42.070624+0000 mgr.y (mgr.14547) 411 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: cluster 2024-02-10T08:32:42.729224+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: audit 2024-02-10T08:32:43.283101+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi067-54245-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: cluster 2024-02-10T08:32:43.283239+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: audit 2024-02-10T08:32:43.288395+0000 mon.c (mon.2) 694 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:44.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:44 smithi134 bash[18722]: audit 2024-02-10T08:32:43.290557+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:44.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: cluster 2024-02-10T08:32:42.032459+0000 mgr.y (mgr.14547) 410 : cluster [DBG] pgmap v552: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: audit 2024-02-10T08:32:42.070624+0000 mgr.y (mgr.14547) 411 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: cluster 2024-02-10T08:32:42.729224+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: audit 2024-02-10T08:32:43.283101+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi067-54245-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: cluster 2024-02-10T08:32:43.283239+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: audit 2024-02-10T08:32:43.288395+0000 mon.c (mon.2) 694 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[17385]: audit 2024-02-10T08:32:43.290557+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: cluster 2024-02-10T08:32:42.032459+0000 mgr.y (mgr.14547) 410 : cluster [DBG] pgmap v552: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: audit 2024-02-10T08:32:42.070624+0000 mgr.y (mgr.14547) 411 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: cluster 2024-02-10T08:32:42.729224+0000 mon.a (mon.0) 2053 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: audit 2024-02-10T08:32:43.283101+0000 mon.a (mon.0) 2054 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi067-54245-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:44.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: cluster 2024-02-10T08:32:43.283239+0000 mon.a (mon.0) 2055 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-10T08:32:44.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: audit 2024-02-10T08:32:43.288395+0000 mon.c (mon.2) 694 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:44.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:44 smithi067 bash[24488]: audit 2024-02-10T08:32:43.290557+0000 mon.a (mon.0) 2056 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:32:45.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[17385]: cluster 2024-02-10T08:32:44.033352+0000 mgr.y (mgr.14547) 412 : cluster [DBG] pgmap v555: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[17385]: audit 2024-02-10T08:32:44.285336+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[17385]: cluster 2024-02-10T08:32:44.285619+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[17385]: audit 2024-02-10T08:32:44.313867+0000 mon.c (mon.2) 695 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[17385]: audit 2024-02-10T08:32:44.314428+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[24488]: cluster 2024-02-10T08:32:44.033352+0000 mgr.y (mgr.14547) 412 : cluster [DBG] pgmap v555: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[24488]: audit 2024-02-10T08:32:44.285336+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[24488]: cluster 2024-02-10T08:32:44.285619+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[24488]: audit 2024-02-10T08:32:44.313867+0000 mon.c (mon.2) 695 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:45 smithi067 bash[24488]: audit 2024-02-10T08:32:44.314428+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:45 smithi134 bash[18722]: cluster 2024-02-10T08:32:44.033352+0000 mgr.y (mgr.14547) 412 : cluster [DBG] pgmap v555: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:32:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:45 smithi134 bash[18722]: audit 2024-02-10T08:32:44.285336+0000 mon.a (mon.0) 2057 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:32:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:45 smithi134 bash[18722]: cluster 2024-02-10T08:32:44.285619+0000 mon.a (mon.0) 2058 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-10T08:32:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:45 smithi134 bash[18722]: audit 2024-02-10T08:32:44.313867+0000 mon.c (mon.2) 695 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:45 smithi134 bash[18722]: audit 2024-02-10T08:32:44.314428+0000 mon.a (mon.0) 2059 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:32:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[17385]: audit 2024-02-10T08:32:45.295184+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:46.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[17385]: cluster 2024-02-10T08:32:45.295321+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-10T08:32:46.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[17385]: audit 2024-02-10T08:32:45.298826+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:46.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[17385]: audit 2024-02-10T08:32:45.299595+0000 mon.c (mon.2) 696 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:46.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[17385]: audit 2024-02-10T08:32:45.300220+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[24488]: audit 2024-02-10T08:32:45.295184+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[24488]: cluster 2024-02-10T08:32:45.295321+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-10T08:32:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[24488]: audit 2024-02-10T08:32:45.298826+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[24488]: audit 2024-02-10T08:32:45.299595+0000 mon.c (mon.2) 696 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:46.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:46 smithi067 bash[24488]: audit 2024-02-10T08:32:45.300220+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:46.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:46 smithi134 bash[18722]: audit 2024-02-10T08:32:45.295184+0000 mon.a (mon.0) 2060 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:32:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:46 smithi134 bash[18722]: cluster 2024-02-10T08:32:45.295321+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-10T08:32:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:46 smithi134 bash[18722]: audit 2024-02-10T08:32:45.298826+0000 mon.a (mon.0) 2062 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:46 smithi134 bash[18722]: audit 2024-02-10T08:32:45.299595+0000 mon.c (mon.2) 696 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:46 smithi134 bash[18722]: audit 2024-02-10T08:32:45.300220+0000 mon.a (mon.0) 2063 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: cluster 2024-02-10T08:32:46.034154+0000 mgr.y (mgr.14547) 413 : cluster [DBG] pgmap v558: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: audit 2024-02-10T08:32:46.310982+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: audit 2024-02-10T08:32:46.311135+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]': finished 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: cluster 2024-02-10T08:32:46.311298+0000 mon.a (mon.0) 2066 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: audit 2024-02-10T08:32:46.313018+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: audit 2024-02-10T08:32:46.317222+0000 mon.c (mon.2) 697 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]: dispatch 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17385]: audit 2024-02-10T08:32:46.317732+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]: dispatch 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: cluster 2024-02-10T08:32:46.034154+0000 mgr.y (mgr.14547) 413 : cluster [DBG] pgmap v558: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: audit 2024-02-10T08:32:46.310982+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: audit 2024-02-10T08:32:46.311135+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]': finished 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: cluster 2024-02-10T08:32:46.311298+0000 mon.a (mon.0) 2066 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-10T08:32:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: audit 2024-02-10T08:32:46.313018+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:47.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: audit 2024-02-10T08:32:46.317222+0000 mon.c (mon.2) 697 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]: dispatch 2024-02-10T08:32:47.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[24488]: audit 2024-02-10T08:32:46.317732+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]: dispatch 2024-02-10T08:32:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: cluster 2024-02-10T08:32:46.034154+0000 mgr.y (mgr.14547) 413 : cluster [DBG] pgmap v558: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: audit 2024-02-10T08:32:46.310982+0000 mon.a (mon.0) 2064 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:47.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: audit 2024-02-10T08:32:46.311135+0000 mon.a (mon.0) 2065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-47"}]': finished 2024-02-10T08:32:47.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: cluster 2024-02-10T08:32:46.311298+0000 mon.a (mon.0) 2066 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-10T08:32:47.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: audit 2024-02-10T08:32:46.313018+0000 mon.a (mon.0) 2067 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]: dispatch 2024-02-10T08:32:47.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: audit 2024-02-10T08:32:46.317222+0000 mon.c (mon.2) 697 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]: dispatch 2024-02-10T08:32:47.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:47 smithi134 bash[18722]: audit 2024-02-10T08:32:46.317732+0000 mon.a (mon.0) 2068 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]: dispatch 2024-02-10T08:32:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:32:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:47] "GET /metrics HTTP/1.1" 200 187990 "" "Prometheus/2.33.4" 2024-02-10T08:32:48.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: cluster 2024-02-10T08:32:47.305986+0000 mon.a (mon.0) 2069 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:32:48.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: audit 2024-02-10T08:32:47.317459+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: audit 2024-02-10T08:32:47.317661+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]': finished 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: cluster 2024-02-10T08:32:47.317851+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: audit 2024-02-10T08:32:47.346674+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: audit 2024-02-10T08:32:47.348247+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[17385]: audit 2024-02-10T08:32:47.348854+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi067-54245-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: cluster 2024-02-10T08:32:47.305986+0000 mon.a (mon.0) 2069 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: audit 2024-02-10T08:32:47.317459+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: audit 2024-02-10T08:32:47.317661+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]': finished 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: cluster 2024-02-10T08:32:47.317851+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: audit 2024-02-10T08:32:47.346674+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:48.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: audit 2024-02-10T08:32:47.348247+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:48.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:48 smithi067 bash[24488]: audit 2024-02-10T08:32:47.348854+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi067-54245-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: cluster 2024-02-10T08:32:47.305986+0000 mon.a (mon.0) 2069 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:32:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: audit 2024-02-10T08:32:47.317459+0000 mon.a (mon.0) 2070 : audit [INF] from='client.? 172.21.15.67:0/2575337025' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi067-54245-61"}]': finished 2024-02-10T08:32:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: audit 2024-02-10T08:32:47.317661+0000 mon.a (mon.0) 2071 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-47", "mode": "readproxy"}]': finished 2024-02-10T08:32:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: cluster 2024-02-10T08:32:47.317851+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-10T08:32:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: audit 2024-02-10T08:32:47.346674+0000 mon.a (mon.0) 2073 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: audit 2024-02-10T08:32:47.348247+0000 mon.a (mon.0) 2074 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:48.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:48 smithi134 bash[18722]: audit 2024-02-10T08:32:47.348854+0000 mon.a (mon.0) 2075 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi067-54245-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[17385]: cluster 2024-02-10T08:32:48.035040+0000 mgr.y (mgr.14547) 414 : cluster [DBG] pgmap v561: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 180 op/s 2024-02-10T08:32:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[17385]: audit 2024-02-10T08:32:48.343882+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi067-54245-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[17385]: cluster 2024-02-10T08:32:48.343949+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-10T08:32:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[17385]: audit 2024-02-10T08:32:48.344671+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi067-54245-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[24488]: cluster 2024-02-10T08:32:48.035040+0000 mgr.y (mgr.14547) 414 : cluster [DBG] pgmap v561: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 180 op/s 2024-02-10T08:32:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[24488]: audit 2024-02-10T08:32:48.343882+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi067-54245-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[24488]: cluster 2024-02-10T08:32:48.343949+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-10T08:32:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:49 smithi067 bash[24488]: audit 2024-02-10T08:32:48.344671+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi067-54245-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:49 smithi134 bash[18722]: cluster 2024-02-10T08:32:48.035040+0000 mgr.y (mgr.14547) 414 : cluster [DBG] pgmap v561: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 180 op/s 2024-02-10T08:32:49.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:49 smithi134 bash[18722]: audit 2024-02-10T08:32:48.343882+0000 mon.a (mon.0) 2076 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi067-54245-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:49.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:49 smithi134 bash[18722]: cluster 2024-02-10T08:32:48.343949+0000 mon.a (mon.0) 2077 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-10T08:32:49.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:49 smithi134 bash[18722]: audit 2024-02-10T08:32:48.344671+0000 mon.a (mon.0) 2078 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi067-54245-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:50.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:50 smithi067 bash[24488]: cluster 2024-02-10T08:32:49.835556+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-10T08:32:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:50 smithi067 bash[17385]: cluster 2024-02-10T08:32:49.835556+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-10T08:32:50.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:50 smithi134 bash[18722]: cluster 2024-02-10T08:32:49.835556+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-10T08:32:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:51 smithi134 bash[18722]: cluster 2024-02-10T08:32:50.035813+0000 mgr.y (mgr.14547) 415 : cluster [DBG] pgmap v564: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 180 op/s 2024-02-10T08:32:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:51 smithi134 bash[18722]: audit 2024-02-10T08:32:50.369991+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi067-54245-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:51 smithi134 bash[18722]: cluster 2024-02-10T08:32:50.370101+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-10T08:32:51.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:51 smithi067 bash[24488]: cluster 2024-02-10T08:32:50.035813+0000 mgr.y (mgr.14547) 415 : cluster [DBG] pgmap v564: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 180 op/s 2024-02-10T08:32:51.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:51 smithi067 bash[24488]: audit 2024-02-10T08:32:50.369991+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi067-54245-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:51.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:51 smithi067 bash[24488]: cluster 2024-02-10T08:32:50.370101+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-10T08:32:51.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:51 smithi067 bash[17385]: cluster 2024-02-10T08:32:50.035813+0000 mgr.y (mgr.14547) 415 : cluster [DBG] pgmap v564: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 108 KiB/s rd, 255 B/s wr, 180 op/s 2024-02-10T08:32:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:51 smithi067 bash[17385]: audit 2024-02-10T08:32:50.369991+0000 mon.a (mon.0) 2080 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi067-54245-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:51 smithi067 bash[17385]: cluster 2024-02-10T08:32:50.370101+0000 mon.a (mon.0) 2081 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-10T08:32:52.397 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:32:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:32:52.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:52 smithi134 bash[18722]: cluster 2024-02-10T08:32:51.372495+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-10T08:32:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:52 smithi067 bash[17385]: cluster 2024-02-10T08:32:51.372495+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-10T08:32:52.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:52 smithi067 bash[24488]: cluster 2024-02-10T08:32:51.372495+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-10T08:32:53.401 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:32:53.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: cluster 2024-02-10T08:32:52.038090+0000 mgr.y (mgr.14547) 416 : cluster [DBG] pgmap v567: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:32:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: audit 2024-02-10T08:32:52.075096+0000 mgr.y (mgr.14547) 417 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: cluster 2024-02-10T08:32:52.391535+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-10T08:32:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: audit 2024-02-10T08:32:52.395117+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: audit 2024-02-10T08:32:53.393346+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: cluster 2024-02-10T08:32:53.393578+0000 mon.a (mon.0) 2086 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-10T08:32:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:53 smithi134 bash[18722]: audit 2024-02-10T08:32:53.394382+0000 mon.a (mon.0) 2087 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: cluster 2024-02-10T08:32:52.038090+0000 mgr.y (mgr.14547) 416 : cluster [DBG] pgmap v567: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:32:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: audit 2024-02-10T08:32:52.075096+0000 mgr.y (mgr.14547) 417 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: cluster 2024-02-10T08:32:52.391535+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-10T08:32:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: audit 2024-02-10T08:32:52.395117+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: audit 2024-02-10T08:32:53.393346+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: cluster 2024-02-10T08:32:53.393578+0000 mon.a (mon.0) 2086 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[17385]: audit 2024-02-10T08:32:53.394382+0000 mon.a (mon.0) 2087 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: cluster 2024-02-10T08:32:52.038090+0000 mgr.y (mgr.14547) 416 : cluster [DBG] pgmap v567: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: audit 2024-02-10T08:32:52.075096+0000 mgr.y (mgr.14547) 417 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: cluster 2024-02-10T08:32:52.391535+0000 mon.a (mon.0) 2083 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: audit 2024-02-10T08:32:52.395117+0000 mon.a (mon.0) 2084 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: audit 2024-02-10T08:32:53.393346+0000 mon.a (mon.0) 2085 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: cluster 2024-02-10T08:32:53.393578+0000 mon.a (mon.0) 2086 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-10T08:32:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:53 smithi067 bash[24488]: audit 2024-02-10T08:32:53.394382+0000 mon.a (mon.0) 2087 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]: dispatch 2024-02-10T08:32:55.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:55 smithi134 bash[18722]: cluster 2024-02-10T08:32:54.038967+0000 mgr.y (mgr.14547) 418 : cluster [DBG] pgmap v570: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T08:32:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:55 smithi134 bash[18722]: audit 2024-02-10T08:32:54.397868+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:55 smithi134 bash[18722]: cluster 2024-02-10T08:32:54.397969+0000 mon.a (mon.0) 2089 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-10T08:32:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:55 smithi134 bash[18722]: audit 2024-02-10T08:32:54.427224+0000 mon.a (mon.0) 2090 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:55 smithi134 bash[18722]: audit 2024-02-10T08:32:54.428993+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:55 smithi134 bash[18722]: audit 2024-02-10T08:32:54.429602+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi067-54245-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[17385]: cluster 2024-02-10T08:32:54.038967+0000 mgr.y (mgr.14547) 418 : cluster [DBG] pgmap v570: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[17385]: audit 2024-02-10T08:32:54.397868+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[17385]: cluster 2024-02-10T08:32:54.397969+0000 mon.a (mon.0) 2089 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[17385]: audit 2024-02-10T08:32:54.427224+0000 mon.a (mon.0) 2090 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[17385]: audit 2024-02-10T08:32:54.428993+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[17385]: audit 2024-02-10T08:32:54.429602+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi067-54245-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[24488]: cluster 2024-02-10T08:32:54.038967+0000 mgr.y (mgr.14547) 418 : cluster [DBG] pgmap v570: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[24488]: audit 2024-02-10T08:32:54.397868+0000 mon.a (mon.0) 2088 : audit [INF] from='client.? 172.21.15.67:0/494387417' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi067-54245-62"}]': finished 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[24488]: cluster 2024-02-10T08:32:54.397969+0000 mon.a (mon.0) 2089 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[24488]: audit 2024-02-10T08:32:54.427224+0000 mon.a (mon.0) 2090 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[24488]: audit 2024-02-10T08:32:54.428993+0000 mon.a (mon.0) 2091 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:55.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:55 smithi067 bash[24488]: audit 2024-02-10T08:32:54.429602+0000 mon.a (mon.0) 2092 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi067-54245-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:32:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:56 smithi134 bash[18722]: audit 2024-02-10T08:32:55.408615+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi067-54245-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:56 smithi134 bash[18722]: cluster 2024-02-10T08:32:55.408744+0000 mon.a (mon.0) 2094 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-10T08:32:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:56 smithi134 bash[18722]: audit 2024-02-10T08:32:55.410677+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi067-54245-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:56 smithi067 bash[17385]: audit 2024-02-10T08:32:55.408615+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi067-54245-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:56 smithi067 bash[17385]: cluster 2024-02-10T08:32:55.408744+0000 mon.a (mon.0) 2094 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-10T08:32:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:56 smithi067 bash[17385]: audit 2024-02-10T08:32:55.410677+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi067-54245-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:56.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:56 smithi067 bash[24488]: audit 2024-02-10T08:32:55.408615+0000 mon.a (mon.0) 2093 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi067-54245-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:32:56.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:56 smithi067 bash[24488]: cluster 2024-02-10T08:32:55.408744+0000 mon.a (mon.0) 2094 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-10T08:32:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:56 smithi067 bash[24488]: audit 2024-02-10T08:32:55.410677+0000 mon.a (mon.0) 2095 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi067-54245-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:57 smithi134 bash[18722]: cluster 2024-02-10T08:32:56.039752+0000 mgr.y (mgr.14547) 419 : cluster [DBG] pgmap v573: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:57 smithi134 bash[18722]: cluster 2024-02-10T08:32:56.871073+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-10T08:32:57.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:57 smithi067 bash[17385]: cluster 2024-02-10T08:32:56.039752+0000 mgr.y (mgr.14547) 419 : cluster [DBG] pgmap v573: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:57 smithi067 bash[17385]: cluster 2024-02-10T08:32:56.871073+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-10T08:32:57.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:57 smithi067 bash[24488]: cluster 2024-02-10T08:32:56.039752+0000 mgr.y (mgr.14547) 419 : cluster [DBG] pgmap v573: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:32:57.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:57 smithi067 bash[24488]: cluster 2024-02-10T08:32:56.871073+0000 mon.a (mon.0) 2096 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-10T08:32:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:32:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:32:57] "GET /metrics HTTP/1.1" 200 187990 "" "Prometheus/2.33.4" 2024-02-10T08:32:58.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:58 smithi134 bash[18722]: audit 2024-02-10T08:32:57.428552+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi067-54245-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:32:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:58 smithi134 bash[18722]: cluster 2024-02-10T08:32:57.428674+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-10T08:32:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:58 smithi134 bash[18722]: audit 2024-02-10T08:32:58.007099+0000 mon.c (mon.2) 698 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:58 smithi134 bash[18722]: audit 2024-02-10T08:32:58.008029+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[17385]: audit 2024-02-10T08:32:57.428552+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi067-54245-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:32:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[17385]: cluster 2024-02-10T08:32:57.428674+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-10T08:32:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[17385]: audit 2024-02-10T08:32:58.007099+0000 mon.c (mon.2) 698 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:58.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[17385]: audit 2024-02-10T08:32:58.008029+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[24488]: audit 2024-02-10T08:32:57.428552+0000 mon.a (mon.0) 2097 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi067-54245-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:32:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[24488]: cluster 2024-02-10T08:32:57.428674+0000 mon.a (mon.0) 2098 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-10T08:32:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[24488]: audit 2024-02-10T08:32:58.007099+0000 mon.c (mon.2) 698 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:58 smithi067 bash[24488]: audit 2024-02-10T08:32:58.008029+0000 mon.a (mon.0) 2099 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: cluster 2024-02-10T08:32:58.040642+0000 mgr.y (mgr.14547) 420 : cluster [DBG] pgmap v576: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: audit 2024-02-10T08:32:58.433807+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: cluster 2024-02-10T08:32:58.433931+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: audit 2024-02-10T08:32:58.435526+0000 mon.c (mon.2) 699 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: audit 2024-02-10T08:32:58.440452+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: cluster 2024-02-10T08:32:59.427878+0000 mon.a (mon.0) 2103 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: audit 2024-02-10T08:32:59.436074+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]': finished 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: cluster 2024-02-10T08:32:59.436257+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-10T08:32:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:32:59 smithi134 bash[18722]: audit 2024-02-10T08:32:59.438681+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: cluster 2024-02-10T08:32:58.040642+0000 mgr.y (mgr.14547) 420 : cluster [DBG] pgmap v576: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: audit 2024-02-10T08:32:58.433807+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: cluster 2024-02-10T08:32:58.433931+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: audit 2024-02-10T08:32:58.435526+0000 mon.c (mon.2) 699 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: audit 2024-02-10T08:32:58.440452+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: cluster 2024-02-10T08:32:59.427878+0000 mon.a (mon.0) 2103 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: audit 2024-02-10T08:32:59.436074+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]': finished 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: cluster 2024-02-10T08:32:59.436257+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[17385]: audit 2024-02-10T08:32:59.438681+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: cluster 2024-02-10T08:32:58.040642+0000 mgr.y (mgr.14547) 420 : cluster [DBG] pgmap v576: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: audit 2024-02-10T08:32:58.433807+0000 mon.a (mon.0) 2100 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:32:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: cluster 2024-02-10T08:32:58.433931+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-10T08:32:59.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: audit 2024-02-10T08:32:58.435526+0000 mon.c (mon.2) 699 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:59.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: audit 2024-02-10T08:32:58.440452+0000 mon.a (mon.0) 2102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:32:59.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: cluster 2024-02-10T08:32:59.427878+0000 mon.a (mon.0) 2103 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:32:59.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: audit 2024-02-10T08:32:59.436074+0000 mon.a (mon.0) 2104 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]': finished 2024-02-10T08:32:59.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: cluster 2024-02-10T08:32:59.436257+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-10T08:32:59.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:32:59 smithi067 bash[24488]: audit 2024-02-10T08:32:59.438681+0000 mon.a (mon.0) 2106 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:33:01.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:00 smithi134 bash[18722]: audit 2024-02-10T08:32:59.485656+0000 mon.c (mon.2) 700 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:00 smithi134 bash[18722]: audit 2024-02-10T08:32:59.486450+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:00 smithi134 bash[18722]: audit 2024-02-10T08:32:59.487705+0000 mon.c (mon.2) 701 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:33:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:00 smithi134 bash[18722]: audit 2024-02-10T08:32:59.488367+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:33:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:00 smithi134 bash[18722]: audit 2024-02-10T08:33:00.443787+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:33:01.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:00 smithi134 bash[18722]: cluster 2024-02-10T08:33:00.444826+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-10T08:33:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[17385]: audit 2024-02-10T08:32:59.485656+0000 mon.c (mon.2) 700 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[17385]: audit 2024-02-10T08:32:59.486450+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[17385]: audit 2024-02-10T08:32:59.487705+0000 mon.c (mon.2) 701 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[17385]: audit 2024-02-10T08:32:59.488367+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[17385]: audit 2024-02-10T08:33:00.443787+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[17385]: cluster 2024-02-10T08:33:00.444826+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[24488]: audit 2024-02-10T08:32:59.485656+0000 mon.c (mon.2) 700 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[24488]: audit 2024-02-10T08:32:59.486450+0000 mon.a (mon.0) 2107 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[24488]: audit 2024-02-10T08:32:59.487705+0000 mon.c (mon.2) 701 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[24488]: audit 2024-02-10T08:32:59.488367+0000 mon.a (mon.0) 2108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-47"}]: dispatch 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[24488]: audit 2024-02-10T08:33:00.443787+0000 mon.a (mon.0) 2109 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:33:01.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:00 smithi067 bash[24488]: cluster 2024-02-10T08:33:00.444826+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-10T08:33:02.276 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:33:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: cluster 2024-02-10T08:33:00.041478+0000 mgr.y (mgr.14547) 421 : cluster [DBG] pgmap v579: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:00.447873+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.445590+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: cluster 2024-02-10T08:33:01.445802+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.452091+0000 mon.c (mon.2) 702 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.458315+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.478073+0000 mon.c (mon.2) 703 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.478762+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.481012+0000 mon.c (mon.2) 704 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.481573+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.482403+0000 mon.c (mon.2) 705 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:02.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:01 smithi134 bash[18722]: audit 2024-02-10T08:33:01.483082+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: cluster 2024-02-10T08:33:00.041478+0000 mgr.y (mgr.14547) 421 : cluster [DBG] pgmap v579: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:00.447873+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.445590+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: cluster 2024-02-10T08:33:01.445802+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.452091+0000 mon.c (mon.2) 702 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.458315+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.478073+0000 mon.c (mon.2) 703 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.478762+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.481012+0000 mon.c (mon.2) 704 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.481573+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.482403+0000 mon.c (mon.2) 705 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[17385]: audit 2024-02-10T08:33:01.483082+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: cluster 2024-02-10T08:33:00.041478+0000 mgr.y (mgr.14547) 421 : cluster [DBG] pgmap v579: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:00.447873+0000 mon.a (mon.0) 2111 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.445590+0000 mon.a (mon.0) 2112 : audit [INF] from='client.? 172.21.15.67:0/1621741181' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi067-54245-63"}]': finished 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: cluster 2024-02-10T08:33:01.445802+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.452091+0000 mon.c (mon.2) 702 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.458315+0000 mon.a (mon.0) 2114 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.478073+0000 mon.c (mon.2) 703 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.478762+0000 mon.a (mon.0) 2115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.481012+0000 mon.c (mon.2) 704 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.481573+0000 mon.a (mon.0) 2116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:02.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.482403+0000 mon.c (mon.2) 705 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:02.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:01 smithi067 bash[24488]: audit 2024-02-10T08:33:01.483082+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: cluster 2024-02-10T08:33:02.043704+0000 mgr.y (mgr.14547) 422 : cluster [DBG] pgmap v582: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.086502+0000 mgr.y (mgr.14547) 423 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.453260+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.455408+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: cluster 2024-02-10T08:33:02.455637+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.456669+0000 mon.c (mon.2) 706 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.458622+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.482830+0000 mon.c (mon.2) 707 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:33:04.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:03 smithi134 bash[18722]: audit 2024-02-10T08:33:02.483208+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: cluster 2024-02-10T08:33:02.043704+0000 mgr.y (mgr.14547) 422 : cluster [DBG] pgmap v582: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.086502+0000 mgr.y (mgr.14547) 423 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.453260+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.455408+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: cluster 2024-02-10T08:33:02.455637+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.456669+0000 mon.c (mon.2) 706 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.458622+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.482830+0000 mon.c (mon.2) 707 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:33:04.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[17385]: audit 2024-02-10T08:33:02.483208+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: cluster 2024-02-10T08:33:02.043704+0000 mgr.y (mgr.14547) 422 : cluster [DBG] pgmap v582: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.086502+0000 mgr.y (mgr.14547) 423 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.453260+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.455408+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi067-54245-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: cluster 2024-02-10T08:33:02.455637+0000 mon.a (mon.0) 2120 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.456669+0000 mon.c (mon.2) 706 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.458622+0000 mon.a (mon.0) 2121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.482830+0000 mon.c (mon.2) 707 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:33:04.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:03 smithi067 bash[24488]: audit 2024-02-10T08:33:02.483208+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:33:05.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:03.902007+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: cluster 2024-02-10T08:33:03.902201+0000 mon.a (mon.0) 2124 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:03.903174+0000 mon.c (mon.2) 708 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:03.904220+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: cluster 2024-02-10T08:33:04.044504+0000 mgr.y (mgr.14547) 424 : cluster [DBG] pgmap v585: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:04.458976+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:04.459182+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]': finished 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: cluster 2024-02-10T08:33:04.459338+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:04.462253+0000 mon.c (mon.2) 709 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]: dispatch 2024-02-10T08:33:05.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:04 smithi134 bash[18722]: audit 2024-02-10T08:33:04.464004+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]: dispatch 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:03.902007+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: cluster 2024-02-10T08:33:03.902201+0000 mon.a (mon.0) 2124 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:03.903174+0000 mon.c (mon.2) 708 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:03.904220+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: cluster 2024-02-10T08:33:04.044504+0000 mgr.y (mgr.14547) 424 : cluster [DBG] pgmap v585: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:04.458976+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:04.459182+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]': finished 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: cluster 2024-02-10T08:33:04.459338+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:04.462253+0000 mon.c (mon.2) 709 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]: dispatch 2024-02-10T08:33:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[17385]: audit 2024-02-10T08:33:04.464004+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]: dispatch 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:03.902007+0000 mon.a (mon.0) 2123 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: cluster 2024-02-10T08:33:03.902201+0000 mon.a (mon.0) 2124 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:03.903174+0000 mon.c (mon.2) 708 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:03.904220+0000 mon.a (mon.0) 2125 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: cluster 2024-02-10T08:33:04.044504+0000 mgr.y (mgr.14547) 424 : cluster [DBG] pgmap v585: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:04.458976+0000 mon.a (mon.0) 2126 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi067-54245-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:04.459182+0000 mon.a (mon.0) 2127 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-49"}]': finished 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: cluster 2024-02-10T08:33:04.459338+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:04.462253+0000 mon.c (mon.2) 709 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]: dispatch 2024-02-10T08:33:05.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:04 smithi067 bash[24488]: audit 2024-02-10T08:33:04.464004+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]: dispatch 2024-02-10T08:33:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:05 smithi134 bash[18722]: cluster 2024-02-10T08:33:05.448960+0000 mon.a (mon.0) 2130 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:33:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:05 smithi134 bash[18722]: audit 2024-02-10T08:33:05.457306+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]': finished 2024-02-10T08:33:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:05 smithi134 bash[18722]: cluster 2024-02-10T08:33:05.457677+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-10T08:33:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:05 smithi134 bash[18722]: audit 2024-02-10T08:33:05.518301+0000 mon.c (mon.2) 710 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:33:06.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:05 smithi134 bash[18722]: audit 2024-02-10T08:33:05.518926+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:33:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[17385]: cluster 2024-02-10T08:33:05.448960+0000 mon.a (mon.0) 2130 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:33:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[17385]: audit 2024-02-10T08:33:05.457306+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]': finished 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[17385]: cluster 2024-02-10T08:33:05.457677+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[17385]: audit 2024-02-10T08:33:05.518301+0000 mon.c (mon.2) 710 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[17385]: audit 2024-02-10T08:33:05.518926+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[24488]: cluster 2024-02-10T08:33:05.448960+0000 mon.a (mon.0) 2130 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[24488]: audit 2024-02-10T08:33:05.457306+0000 mon.a (mon.0) 2131 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-49", "mode": "writeback"}]': finished 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[24488]: cluster 2024-02-10T08:33:05.457677+0000 mon.a (mon.0) 2132 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[24488]: audit 2024-02-10T08:33:05.518301+0000 mon.c (mon.2) 710 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:33:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:05 smithi067 bash[24488]: audit 2024-02-10T08:33:05.518926+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:33:07.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: cluster 2024-02-10T08:33:06.045389+0000 mgr.y (mgr.14547) 425 : cluster [DBG] pgmap v588: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: audit 2024-02-10T08:33:06.460230+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:33:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: cluster 2024-02-10T08:33:06.460474+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-10T08:33:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: audit 2024-02-10T08:33:06.462317+0000 mon.c (mon.2) 711 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:33:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: audit 2024-02-10T08:33:06.462734+0000 mon.c (mon.2) 712 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: audit 2024-02-10T08:33:06.463002+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:33:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:07 smithi134 bash[18722]: audit 2024-02-10T08:33:06.463775+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: cluster 2024-02-10T08:33:06.045389+0000 mgr.y (mgr.14547) 425 : cluster [DBG] pgmap v588: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: audit 2024-02-10T08:33:06.460230+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: cluster 2024-02-10T08:33:06.460474+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: audit 2024-02-10T08:33:06.462317+0000 mon.c (mon.2) 711 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: audit 2024-02-10T08:33:06.462734+0000 mon.c (mon.2) 712 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: audit 2024-02-10T08:33:06.463002+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17385]: audit 2024-02-10T08:33:06.463775+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: cluster 2024-02-10T08:33:06.045389+0000 mgr.y (mgr.14547) 425 : cluster [DBG] pgmap v588: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: audit 2024-02-10T08:33:06.460230+0000 mon.a (mon.0) 2134 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: cluster 2024-02-10T08:33:06.460474+0000 mon.a (mon.0) 2135 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-10T08:33:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: audit 2024-02-10T08:33:06.462317+0000 mon.c (mon.2) 711 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:33:07.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: audit 2024-02-10T08:33:06.462734+0000 mon.c (mon.2) 712 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:07.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: audit 2024-02-10T08:33:06.463002+0000 mon.a (mon.0) 2136 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:33:07.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[24488]: audit 2024-02-10T08:33:06.463775+0000 mon.a (mon.0) 2137 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:33:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:07] "GET /metrics HTTP/1.1" 200 185579 "" "Prometheus/2.33.4" 2024-02-10T08:33:08.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: audit 2024-02-10T08:33:07.462921+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:33:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: audit 2024-02-10T08:33:07.463237+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: cluster 2024-02-10T08:33:07.463505+0000 mon.a (mon.0) 2140 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-10T08:33:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: audit 2024-02-10T08:33:07.467357+0000 mon.c (mon.2) 713 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:33:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: audit 2024-02-10T08:33:07.467948+0000 mon.c (mon.2) 714 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: audit 2024-02-10T08:33:07.474407+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:33:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:08 smithi134 bash[18722]: audit 2024-02-10T08:33:07.474882+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: audit 2024-02-10T08:33:07.462921+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:33:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: audit 2024-02-10T08:33:07.463237+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: cluster 2024-02-10T08:33:07.463505+0000 mon.a (mon.0) 2140 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: audit 2024-02-10T08:33:07.467357+0000 mon.c (mon.2) 713 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: audit 2024-02-10T08:33:07.467948+0000 mon.c (mon.2) 714 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: audit 2024-02-10T08:33:07.474407+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[17385]: audit 2024-02-10T08:33:07.474882+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: audit 2024-02-10T08:33:07.462921+0000 mon.a (mon.0) 2138 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: audit 2024-02-10T08:33:07.463237+0000 mon.a (mon.0) 2139 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: cluster 2024-02-10T08:33:07.463505+0000 mon.a (mon.0) 2140 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: audit 2024-02-10T08:33:07.467357+0000 mon.c (mon.2) 713 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: audit 2024-02-10T08:33:07.467948+0000 mon.c (mon.2) 714 : audit [INF] from='client.? 172.21.15.67:0/2389636437' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: audit 2024-02-10T08:33:07.474407+0000 mon.a (mon.0) 2141 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:33:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:08 smithi067 bash[24488]: audit 2024-02-10T08:33:07.474882+0000 mon.a (mon.0) 2142 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]: dispatch 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: cluster 2024-02-10T08:33:08.046192+0000 mgr.y (mgr.14547) 426 : cluster [DBG] pgmap v591: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: cluster 2024-02-10T08:33:08.469197+0000 mon.a (mon.0) 2143 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.478075+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.478333+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: cluster 2024-02-10T08:33:08.478453+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.480683+0000 mon.c (mon.2) 715 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.482473+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.508835+0000 mon.c (mon.2) 716 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.509442+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.510415+0000 mon.c (mon.2) 717 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.510931+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.511833+0000 mon.c (mon.2) 718 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:09.778 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:09 smithi134 bash[18722]: audit 2024-02-10T08:33:08.512293+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:09.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: cluster 2024-02-10T08:33:08.046192+0000 mgr.y (mgr.14547) 426 : cluster [DBG] pgmap v591: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:33:09.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: cluster 2024-02-10T08:33:08.469197+0000 mon.a (mon.0) 2143 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.478075+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.478333+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: cluster 2024-02-10T08:33:08.478453+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.480683+0000 mon.c (mon.2) 715 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.482473+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.508835+0000 mon.c (mon.2) 716 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.509442+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.510415+0000 mon.c (mon.2) 717 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.510931+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.511833+0000 mon.c (mon.2) 718 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[24488]: audit 2024-02-10T08:33:08.512293+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: cluster 2024-02-10T08:33:08.046192+0000 mgr.y (mgr.14547) 426 : cluster [DBG] pgmap v591: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:33:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: cluster 2024-02-10T08:33:08.469197+0000 mon.a (mon.0) 2143 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.478075+0000 mon.a (mon.0) 2144 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.478333+0000 mon.a (mon.0) 2145 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi067-54245-64"}]': finished 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: cluster 2024-02-10T08:33:08.478453+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.480683+0000 mon.c (mon.2) 715 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.482473+0000 mon.a (mon.0) 2147 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.508835+0000 mon.c (mon.2) 716 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.509442+0000 mon.a (mon.0) 2148 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.510415+0000 mon.c (mon.2) 717 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.510931+0000 mon.a (mon.0) 2149 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.511833+0000 mon.c (mon.2) 718 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:09.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:09 smithi067 bash[17385]: audit 2024-02-10T08:33:08.512293+0000 mon.a (mon.0) 2150 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: audit 2024-02-10T08:33:09.482601+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: audit 2024-02-10T08:33:09.482749+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: cluster 2024-02-10T08:33:09.482835+0000 mon.a (mon.0) 2153 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: audit 2024-02-10T08:33:09.486659+0000 mon.c (mon.2) 719 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: audit 2024-02-10T08:33:09.486923+0000 mon.c (mon.2) 720 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: audit 2024-02-10T08:33:09.496545+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:33:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:10 smithi134 bash[18722]: audit 2024-02-10T08:33:09.497040+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: audit 2024-02-10T08:33:09.482601+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: audit 2024-02-10T08:33:09.482749+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: cluster 2024-02-10T08:33:09.482835+0000 mon.a (mon.0) 2153 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: audit 2024-02-10T08:33:09.486659+0000 mon.c (mon.2) 719 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: audit 2024-02-10T08:33:09.486923+0000 mon.c (mon.2) 720 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: audit 2024-02-10T08:33:09.496545+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:33:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[17385]: audit 2024-02-10T08:33:09.497040+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: audit 2024-02-10T08:33:09.482601+0000 mon.a (mon.0) 2151 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: audit 2024-02-10T08:33:09.482749+0000 mon.a (mon.0) 2152 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi067-54245-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: cluster 2024-02-10T08:33:09.482835+0000 mon.a (mon.0) 2153 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: audit 2024-02-10T08:33:09.486659+0000 mon.c (mon.2) 719 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: audit 2024-02-10T08:33:09.486923+0000 mon.c (mon.2) 720 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: audit 2024-02-10T08:33:09.496545+0000 mon.a (mon.0) 2154 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:33:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:10 smithi067 bash[24488]: audit 2024-02-10T08:33:09.497040+0000 mon.a (mon.0) 2155 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:11.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:11 smithi134 bash[18722]: cluster 2024-02-10T08:33:10.047094+0000 mgr.y (mgr.14547) 427 : cluster [DBG] pgmap v594: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:33:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:11 smithi134 bash[18722]: audit 2024-02-10T08:33:10.504360+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]': finished 2024-02-10T08:33:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:11 smithi134 bash[18722]: cluster 2024-02-10T08:33:10.961127+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-10T08:33:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:11 smithi067 bash[17385]: cluster 2024-02-10T08:33:10.047094+0000 mgr.y (mgr.14547) 427 : cluster [DBG] pgmap v594: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:33:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:11 smithi067 bash[17385]: audit 2024-02-10T08:33:10.504360+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]': finished 2024-02-10T08:33:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:11 smithi067 bash[17385]: cluster 2024-02-10T08:33:10.961127+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-10T08:33:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:11 smithi067 bash[24488]: cluster 2024-02-10T08:33:10.047094+0000 mgr.y (mgr.14547) 427 : cluster [DBG] pgmap v594: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:33:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:11 smithi067 bash[24488]: audit 2024-02-10T08:33:10.504360+0000 mon.a (mon.0) 2156 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-49","var": "target_max_objects","val": "1"}]': finished 2024-02-10T08:33:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:11 smithi067 bash[24488]: cluster 2024-02-10T08:33:10.961127+0000 mon.a (mon.0) 2157 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-10T08:33:12.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:33:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:33:12.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:12 smithi067 bash[17385]: audit 2024-02-10T08:33:11.516691+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:12.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:12 smithi067 bash[17385]: cluster 2024-02-10T08:33:11.516851+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-10T08:33:12.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:12 smithi067 bash[24488]: audit 2024-02-10T08:33:11.516691+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:12.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:12 smithi067 bash[24488]: cluster 2024-02-10T08:33:11.516851+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-10T08:33:13.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:12 smithi134 bash[18722]: audit 2024-02-10T08:33:11.516691+0000 mon.a (mon.0) 2158 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi067-54245-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:13.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:12 smithi134 bash[18722]: cluster 2024-02-10T08:33:11.516851+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-10T08:33:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:33:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:33:13.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[17385]: cluster 2024-02-10T08:33:12.049420+0000 mgr.y (mgr.14547) 428 : cluster [DBG] pgmap v597: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[17385]: audit 2024-02-10T08:33:12.091010+0000 mgr.y (mgr.14547) 429 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[17385]: cluster 2024-02-10T08:33:12.513784+0000 mon.a (mon.0) 2160 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[17385]: cluster 2024-02-10T08:33:12.519874+0000 mon.a (mon.0) 2161 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[24488]: cluster 2024-02-10T08:33:12.049420+0000 mgr.y (mgr.14547) 428 : cluster [DBG] pgmap v597: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[24488]: audit 2024-02-10T08:33:12.091010+0000 mgr.y (mgr.14547) 429 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[24488]: cluster 2024-02-10T08:33:12.513784+0000 mon.a (mon.0) 2160 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T08:33:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:13 smithi067 bash[24488]: cluster 2024-02-10T08:33:12.519874+0000 mon.a (mon.0) 2161 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-10T08:33:14.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:13 smithi134 bash[18722]: cluster 2024-02-10T08:33:12.049420+0000 mgr.y (mgr.14547) 428 : cluster [DBG] pgmap v597: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:13 smithi134 bash[18722]: audit 2024-02-10T08:33:12.091010+0000 mgr.y (mgr.14547) 429 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:13 smithi134 bash[18722]: cluster 2024-02-10T08:33:12.513784+0000 mon.a (mon.0) 2160 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T08:33:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:13 smithi134 bash[18722]: cluster 2024-02-10T08:33:12.519874+0000 mon.a (mon.0) 2161 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-10T08:33:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:14 smithi067 bash[24488]: cluster 2024-02-10T08:33:13.535054+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-10T08:33:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:14 smithi067 bash[24488]: audit 2024-02-10T08:33:13.538214+0000 mon.c (mon.2) 721 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:14 smithi067 bash[24488]: audit 2024-02-10T08:33:13.544843+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:14 smithi067 bash[17385]: cluster 2024-02-10T08:33:13.535054+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-10T08:33:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:14 smithi067 bash[17385]: audit 2024-02-10T08:33:13.538214+0000 mon.c (mon.2) 721 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:14 smithi067 bash[17385]: audit 2024-02-10T08:33:13.544843+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:14 smithi134 bash[18722]: cluster 2024-02-10T08:33:13.535054+0000 mon.a (mon.0) 2162 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-10T08:33:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:14 smithi134 bash[18722]: audit 2024-02-10T08:33:13.538214+0000 mon.c (mon.2) 721 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:14 smithi134 bash[18722]: audit 2024-02-10T08:33:13.544843+0000 mon.a (mon.0) 2163 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:15.575 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7047 ms) 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7080 ms) 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7057 ms) 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7028 ms) 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7084 ms) 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139811 ms total) 2024-02-10T08:33:15.576 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: 2024-02-10T08:33:15.577 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-10T08:33:15.577 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [==========] 56 tests from 4 test suites ran. (298804 ms total) 2024-02-10T08:33:15.577 INFO:tasks.workunit.client.0.smithi067.stdout: api_aio_pp: [ PASSED ] 56 tests. 2024-02-10T08:33:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[17385]: cluster 2024-02-10T08:33:14.050300+0000 mgr.y (mgr.14547) 430 : cluster [DBG] pgmap v600: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[17385]: audit 2024-02-10T08:33:14.546380+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[17385]: cluster 2024-02-10T08:33:14.546576+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[17385]: audit 2024-02-10T08:33:14.550506+0000 mon.c (mon.2) 722 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[17385]: audit 2024-02-10T08:33:14.556277+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[24488]: cluster 2024-02-10T08:33:14.050300+0000 mgr.y (mgr.14547) 430 : cluster [DBG] pgmap v600: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[24488]: audit 2024-02-10T08:33:14.546380+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[24488]: cluster 2024-02-10T08:33:14.546576+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[24488]: audit 2024-02-10T08:33:14.550506+0000 mon.c (mon.2) 722 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:15 smithi067 bash[24488]: audit 2024-02-10T08:33:14.556277+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:15 smithi134 bash[18722]: cluster 2024-02-10T08:33:14.050300+0000 mgr.y (mgr.14547) 430 : cluster [DBG] pgmap v600: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:15 smithi134 bash[18722]: audit 2024-02-10T08:33:14.546380+0000 mon.a (mon.0) 2164 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:15 smithi134 bash[18722]: cluster 2024-02-10T08:33:14.546576+0000 mon.a (mon.0) 2165 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-10T08:33:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:15 smithi134 bash[18722]: audit 2024-02-10T08:33:14.550506+0000 mon.c (mon.2) 722 : audit [INF] from='client.? 172.21.15.67:0/2467534303' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:15 smithi134 bash[18722]: audit 2024-02-10T08:33:14.556277+0000 mon.a (mon.0) 2166 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]: dispatch 2024-02-10T08:33:16.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:16 smithi067 bash[17385]: audit 2024-02-10T08:33:15.568675+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:16.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:16 smithi067 bash[17385]: cluster 2024-02-10T08:33:15.568784+0000 mon.a (mon.0) 2168 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-10T08:33:16.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:16 smithi067 bash[17385]: audit 2024-02-10T08:33:15.781395+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:16.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:16 smithi067 bash[24488]: audit 2024-02-10T08:33:15.568675+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:16.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:16 smithi067 bash[24488]: cluster 2024-02-10T08:33:15.568784+0000 mon.a (mon.0) 2168 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-10T08:33:16.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:16 smithi067 bash[24488]: audit 2024-02-10T08:33:15.781395+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:17.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:16 smithi134 bash[18722]: audit 2024-02-10T08:33:15.568675+0000 mon.a (mon.0) 2167 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi067-54245-65"}]': finished 2024-02-10T08:33:17.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:16 smithi134 bash[18722]: cluster 2024-02-10T08:33:15.568784+0000 mon.a (mon.0) 2168 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-10T08:33:17.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:16 smithi134 bash[18722]: audit 2024-02-10T08:33:15.781395+0000 mon.a (mon.0) 2169 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:17.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:17 smithi067 bash[17385]: cluster 2024-02-10T08:33:16.050896+0000 mgr.y (mgr.14547) 431 : cluster [DBG] pgmap v603: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:17.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:17 smithi067 bash[24488]: cluster 2024-02-10T08:33:16.050896+0000 mgr.y (mgr.14547) 431 : cluster [DBG] pgmap v603: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:18.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:17 smithi134 bash[18722]: cluster 2024-02-10T08:33:16.050896+0000 mgr.y (mgr.14547) 431 : cluster [DBG] pgmap v603: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:33:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:17] "GET /metrics HTTP/1.1" 200 188227 "" "Prometheus/2.33.4" 2024-02-10T08:33:19.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:19 smithi067 bash[17385]: cluster 2024-02-10T08:33:18.052676+0000 mgr.y (mgr.14547) 432 : cluster [DBG] pgmap v604: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 925 B/s rd, 0 op/s 2024-02-10T08:33:19.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:19 smithi067 bash[24488]: cluster 2024-02-10T08:33:18.052676+0000 mgr.y (mgr.14547) 432 : cluster [DBG] pgmap v604: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 925 B/s rd, 0 op/s 2024-02-10T08:33:20.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:19 smithi134 bash[18722]: cluster 2024-02-10T08:33:18.052676+0000 mgr.y (mgr.14547) 432 : cluster [DBG] pgmap v604: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 925 B/s rd, 0 op/s 2024-02-10T08:33:20.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:20 smithi067 bash[17385]: audit 2024-02-10T08:33:20.533396+0000 mon.c (mon.2) 723 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:20.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:20 smithi067 bash[17385]: audit 2024-02-10T08:33:20.534103+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:20.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:20 smithi067 bash[24488]: audit 2024-02-10T08:33:20.533396+0000 mon.c (mon.2) 723 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:20.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:20 smithi067 bash[24488]: audit 2024-02-10T08:33:20.534103+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:21.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:20 smithi134 bash[18722]: audit 2024-02-10T08:33:20.533396+0000 mon.c (mon.2) 723 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:20 smithi134 bash[18722]: audit 2024-02-10T08:33:20.534103+0000 mon.a (mon.0) 2170 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:21.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[17385]: cluster 2024-02-10T08:33:20.053518+0000 mgr.y (mgr.14547) 433 : cluster [DBG] pgmap v605: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-02-10T08:33:21.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[17385]: audit 2024-02-10T08:33:20.587787+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:33:21.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[17385]: cluster 2024-02-10T08:33:20.587873+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-10T08:33:21.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[17385]: audit 2024-02-10T08:33:20.591059+0000 mon.c (mon.2) 724 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:21.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[17385]: audit 2024-02-10T08:33:20.591748+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:21.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[24488]: cluster 2024-02-10T08:33:20.053518+0000 mgr.y (mgr.14547) 433 : cluster [DBG] pgmap v605: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-02-10T08:33:21.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[24488]: audit 2024-02-10T08:33:20.587787+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:33:21.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[24488]: cluster 2024-02-10T08:33:20.587873+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-10T08:33:21.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[24488]: audit 2024-02-10T08:33:20.591059+0000 mon.c (mon.2) 724 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:21.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:21 smithi067 bash[24488]: audit 2024-02-10T08:33:20.591748+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:22.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:21 smithi134 bash[18722]: cluster 2024-02-10T08:33:20.053518+0000 mgr.y (mgr.14547) 433 : cluster [DBG] pgmap v605: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 785 B/s rd, 0 op/s 2024-02-10T08:33:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:21 smithi134 bash[18722]: audit 2024-02-10T08:33:20.587787+0000 mon.a (mon.0) 2171 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:33:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:21 smithi134 bash[18722]: cluster 2024-02-10T08:33:20.587873+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-10T08:33:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:21 smithi134 bash[18722]: audit 2024-02-10T08:33:20.591059+0000 mon.c (mon.2) 724 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:22.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:21 smithi134 bash[18722]: audit 2024-02-10T08:33:20.591748+0000 mon.a (mon.0) 2173 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:22.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:33:22.620 INFO:tasks.workunit.client.0.smithi067.stdout:4c720e9:::166:head 2024-02-10T08:33:22.620 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e694b040:::167:head 2024-02-10T08:33:22.620 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:afa38db2:::168:head 2024-02-10T08:33:22.620 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:77ba9f53:::169:head 2024-02-10T08:33:22.620 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:87495034:::170:head 2024-02-10T08:33:22.620 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7c96bf0e:::171:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:dbe346cc:::172:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:e943ec24:::173:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f97a9c0c:::174:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:6f26e74d:::175:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4f95e106:::176:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:0e6f2f8f:::177:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:05db05f1:::178:head 2024-02-10T08:33:22.621 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:38a78d66:::179:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:d095610b:::180:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:a1a9d709:::181:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:1e5d39db:::182:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:f7df4fb9:::183:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:03a7f161:::184:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ba70721e:::185:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:28e5662d:::186:head 2024-02-10T08:33:22.622 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:973d52de:::187:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:4303eb1c:::188:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:b990b48e:::189:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:29b8165b:::190:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:3547f197:::191:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:7e260936:::192:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:1abec7b1:::193:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:10fdda93:::194:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:15817eea:::195:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:770bab57:::196:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:ed9e13e7:::197:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:71471a8f:::198:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: checking for 191:10fb1d02:::199:head 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8063 ms) 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,0 2024-02-10T08:33:22.623 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: first is 1707553936 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,1707553941,1707553942,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,1707553941,1707553942,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,1707553941,1707553942,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,1707553941,1707553942,1707553944,1707553945,0 2024-02-10T08:33:22.624 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,1707553941,1707553942,1707553944,1707553945,0 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553936,1707553938,1707553939,1707553941,1707553942,1707553944,1707553945,0 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707553939,1707553941,1707553942,1707553944,1707553945,1707553947,1707553948,0 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: first now 1707553939, trimmed 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20171 ms) 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: foo0 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14563 ms) 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (17719 ms) 2024-02-10T08:33:22.625 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-10T08:33:22.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[17385]: audit 2024-02-10T08:33:21.599112+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]': finished 2024-02-10T08:33:22.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[17385]: cluster 2024-02-10T08:33:21.599632+0000 mon.a (mon.0) 2175 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[17385]: audit 2024-02-10T08:33:21.638894+0000 mon.c (mon.2) 725 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[17385]: audit 2024-02-10T08:33:21.639449+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[17385]: audit 2024-02-10T08:33:21.640298+0000 mon.c (mon.2) 726 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[17385]: audit 2024-02-10T08:33:21.640692+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[24488]: audit 2024-02-10T08:33:21.599112+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]': finished 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[24488]: cluster 2024-02-10T08:33:21.599632+0000 mon.a (mon.0) 2175 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[24488]: audit 2024-02-10T08:33:21.638894+0000 mon.c (mon.2) 725 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[24488]: audit 2024-02-10T08:33:21.639449+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[24488]: audit 2024-02-10T08:33:21.640298+0000 mon.c (mon.2) 726 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:22.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:22 smithi067 bash[24488]: audit 2024-02-10T08:33:21.640692+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[18722]: audit 2024-02-10T08:33:21.599112+0000 mon.a (mon.0) 2174 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]': finished 2024-02-10T08:33:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[18722]: cluster 2024-02-10T08:33:21.599632+0000 mon.a (mon.0) 2175 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-10T08:33:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[18722]: audit 2024-02-10T08:33:21.638894+0000 mon.c (mon.2) 725 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[18722]: audit 2024-02-10T08:33:21.639449+0000 mon.a (mon.0) 2176 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[18722]: audit 2024-02-10T08:33:21.640298+0000 mon.c (mon.2) 726 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:23.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:22 smithi134 bash[18722]: audit 2024-02-10T08:33:21.640692+0000 mon.a (mon.0) 2177 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-49"}]: dispatch 2024-02-10T08:33:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:33:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:33:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[17385]: cluster 2024-02-10T08:33:22.055778+0000 mgr.y (mgr.14547) 434 : cluster [DBG] pgmap v608: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:33:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[17385]: audit 2024-02-10T08:33:22.093875+0000 mgr.y (mgr.14547) 435 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[17385]: cluster 2024-02-10T08:33:22.594557+0000 mon.a (mon.0) 2178 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T08:33:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[17385]: cluster 2024-02-10T08:33:22.613099+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-10T08:33:23.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[24488]: cluster 2024-02-10T08:33:22.055778+0000 mgr.y (mgr.14547) 434 : cluster [DBG] pgmap v608: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:33:23.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[24488]: audit 2024-02-10T08:33:22.093875+0000 mgr.y (mgr.14547) 435 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:23.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[24488]: cluster 2024-02-10T08:33:22.594557+0000 mon.a (mon.0) 2178 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T08:33:23.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:23 smithi067 bash[24488]: cluster 2024-02-10T08:33:22.613099+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-10T08:33:24.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:23 smithi134 bash[18722]: cluster 2024-02-10T08:33:22.055778+0000 mgr.y (mgr.14547) 434 : cluster [DBG] pgmap v608: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:33:24.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:23 smithi134 bash[18722]: audit 2024-02-10T08:33:22.093875+0000 mgr.y (mgr.14547) 435 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:24.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:23 smithi134 bash[18722]: cluster 2024-02-10T08:33:22.594557+0000 mon.a (mon.0) 2178 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T08:33:24.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:23 smithi134 bash[18722]: cluster 2024-02-10T08:33:22.613099+0000 mon.a (mon.0) 2179 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-10T08:33:25.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:24 smithi134 bash[18722]: cluster 2024-02-10T08:33:23.622546+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-10T08:33:25.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:24 smithi134 bash[18722]: audit 2024-02-10T08:33:23.625267+0000 mon.c (mon.2) 727 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:25.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:24 smithi134 bash[18722]: audit 2024-02-10T08:33:23.628614+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:24 smithi067 bash[17385]: cluster 2024-02-10T08:33:23.622546+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-10T08:33:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:24 smithi067 bash[17385]: audit 2024-02-10T08:33:23.625267+0000 mon.c (mon.2) 727 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:25.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:24 smithi067 bash[17385]: audit 2024-02-10T08:33:23.628614+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:24 smithi067 bash[24488]: cluster 2024-02-10T08:33:23.622546+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-10T08:33:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:24 smithi067 bash[24488]: audit 2024-02-10T08:33:23.625267+0000 mon.c (mon.2) 727 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:24 smithi067 bash[24488]: audit 2024-02-10T08:33:23.628614+0000 mon.a (mon.0) 2181 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: cluster 2024-02-10T08:33:24.056587+0000 mgr.y (mgr.14547) 436 : cluster [DBG] pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T08:33:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: audit 2024-02-10T08:33:24.632892+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: cluster 2024-02-10T08:33:24.633153+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-10T08:33:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: audit 2024-02-10T08:33:24.676476+0000 mon.c (mon.2) 728 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: audit 2024-02-10T08:33:24.677012+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: audit 2024-02-10T08:33:24.678108+0000 mon.c (mon.2) 729 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-51"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[17385]: audit 2024-02-10T08:33:24.678652+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-51"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: cluster 2024-02-10T08:33:24.056587+0000 mgr.y (mgr.14547) 436 : cluster [DBG] pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: audit 2024-02-10T08:33:24.632892+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: cluster 2024-02-10T08:33:24.633153+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: audit 2024-02-10T08:33:24.676476+0000 mon.c (mon.2) 728 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: audit 2024-02-10T08:33:24.677012+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: audit 2024-02-10T08:33:24.678108+0000 mon.c (mon.2) 729 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-51"}]: dispatch 2024-02-10T08:33:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:26 smithi067 bash[24488]: audit 2024-02-10T08:33:24.678652+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-51"}]: dispatch 2024-02-10T08:33:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: cluster 2024-02-10T08:33:24.056587+0000 mgr.y (mgr.14547) 436 : cluster [DBG] pgmap v611: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-10T08:33:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: audit 2024-02-10T08:33:24.632892+0000 mon.a (mon.0) 2182 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: cluster 2024-02-10T08:33:24.633153+0000 mon.a (mon.0) 2183 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-10T08:33:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: audit 2024-02-10T08:33:24.676476+0000 mon.c (mon.2) 728 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: audit 2024-02-10T08:33:24.677012+0000 mon.a (mon.0) 2184 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: audit 2024-02-10T08:33:24.678108+0000 mon.c (mon.2) 729 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-51"}]: dispatch 2024-02-10T08:33:26.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:26 smithi134 bash[18722]: audit 2024-02-10T08:33:24.678652+0000 mon.a (mon.0) 2185 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-51"}]: dispatch 2024-02-10T08:33:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[17385]: cluster 2024-02-10T08:33:25.636825+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-10T08:33:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[17385]: cluster 2024-02-10T08:33:26.057287+0000 mgr.y (mgr.14547) 437 : cluster [DBG] pgmap v614: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[17385]: cluster 2024-02-10T08:33:26.642417+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[17385]: audit 2024-02-10T08:33:26.648114+0000 mon.c (mon.2) 730 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[17385]: audit 2024-02-10T08:33:26.649684+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[24488]: cluster 2024-02-10T08:33:25.636825+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[24488]: cluster 2024-02-10T08:33:26.057287+0000 mgr.y (mgr.14547) 437 : cluster [DBG] pgmap v614: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[24488]: cluster 2024-02-10T08:33:26.642417+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[24488]: audit 2024-02-10T08:33:26.648114+0000 mon.c (mon.2) 730 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[24488]: audit 2024-02-10T08:33:26.649684+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:27.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:27 smithi134 bash[18722]: cluster 2024-02-10T08:33:25.636825+0000 mon.a (mon.0) 2186 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-10T08:33:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:27 smithi134 bash[18722]: cluster 2024-02-10T08:33:26.057287+0000 mgr.y (mgr.14547) 437 : cluster [DBG] pgmap v614: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:27 smithi134 bash[18722]: cluster 2024-02-10T08:33:26.642417+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-10T08:33:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:27 smithi134 bash[18722]: audit 2024-02-10T08:33:26.648114+0000 mon.c (mon.2) 730 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:27 smithi134 bash[18722]: audit 2024-02-10T08:33:26.649684+0000 mon.a (mon.0) 2188 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:33:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:27] "GET /metrics HTTP/1.1" 200 188227 "" "Prometheus/2.33.4" 2024-02-10T08:33:29.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: audit 2024-02-10T08:33:27.644842+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: cluster 2024-02-10T08:33:27.644974+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-10T08:33:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: audit 2024-02-10T08:33:27.648270+0000 mon.c (mon.2) 731 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: audit 2024-02-10T08:33:27.710262+0000 mon.c (mon.2) 732 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: audit 2024-02-10T08:33:27.711090+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: audit 2024-02-10T08:33:27.712339+0000 mon.c (mon.2) 733 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-53"}]: dispatch 2024-02-10T08:33:29.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:28 smithi134 bash[18722]: audit 2024-02-10T08:33:27.713035+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-53"}]: dispatch 2024-02-10T08:33:29.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: audit 2024-02-10T08:33:27.644842+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:29.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: cluster 2024-02-10T08:33:27.644974+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: audit 2024-02-10T08:33:27.648270+0000 mon.c (mon.2) 731 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: audit 2024-02-10T08:33:27.710262+0000 mon.c (mon.2) 732 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: audit 2024-02-10T08:33:27.711090+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: audit 2024-02-10T08:33:27.712339+0000 mon.c (mon.2) 733 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-53"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[17385]: audit 2024-02-10T08:33:27.713035+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-53"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: audit 2024-02-10T08:33:27.644842+0000 mon.a (mon.0) 2189 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: cluster 2024-02-10T08:33:27.644974+0000 mon.a (mon.0) 2190 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: audit 2024-02-10T08:33:27.648270+0000 mon.c (mon.2) 731 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: audit 2024-02-10T08:33:27.710262+0000 mon.c (mon.2) 732 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: audit 2024-02-10T08:33:27.711090+0000 mon.a (mon.0) 2191 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: audit 2024-02-10T08:33:27.712339+0000 mon.c (mon.2) 733 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-53"}]: dispatch 2024-02-10T08:33:29.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:28 smithi067 bash[24488]: audit 2024-02-10T08:33:27.713035+0000 mon.a (mon.0) 2192 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-53"}]: dispatch 2024-02-10T08:33:30.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:29 smithi134 bash[18722]: cluster 2024-02-10T08:33:28.058214+0000 mgr.y (mgr.14547) 438 : cluster [DBG] pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:29 smithi134 bash[18722]: cluster 2024-02-10T08:33:28.652648+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-10T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:29 smithi067 bash[17385]: cluster 2024-02-10T08:33:28.058214+0000 mgr.y (mgr.14547) 438 : cluster [DBG] pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:29 smithi067 bash[17385]: cluster 2024-02-10T08:33:28.652648+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-10T08:33:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:29 smithi067 bash[24488]: cluster 2024-02-10T08:33:28.058214+0000 mgr.y (mgr.14547) 438 : cluster [DBG] pgmap v617: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:29 smithi067 bash[24488]: cluster 2024-02-10T08:33:28.652648+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-10T08:33:31.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:30 smithi134 bash[18722]: cluster 2024-02-10T08:33:29.662590+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-10T08:33:31.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:30 smithi134 bash[18722]: audit 2024-02-10T08:33:29.666164+0000 mon.c (mon.2) 734 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:31.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:30 smithi134 bash[18722]: audit 2024-02-10T08:33:29.667369+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:31.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:30 smithi067 bash[24488]: cluster 2024-02-10T08:33:29.662590+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-10T08:33:31.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:30 smithi067 bash[24488]: audit 2024-02-10T08:33:29.666164+0000 mon.c (mon.2) 734 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:31.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:30 smithi067 bash[24488]: audit 2024-02-10T08:33:29.667369+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:30 smithi067 bash[17385]: cluster 2024-02-10T08:33:29.662590+0000 mon.a (mon.0) 2194 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-10T08:33:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:30 smithi067 bash[17385]: audit 2024-02-10T08:33:29.666164+0000 mon.c (mon.2) 734 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:31.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:30 smithi067 bash[17385]: audit 2024-02-10T08:33:29.667369+0000 mon.a (mon.0) 2195 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:32.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: cluster 2024-02-10T08:33:30.059248+0000 mgr.y (mgr.14547) 439 : cluster [DBG] pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: audit 2024-02-10T08:33:30.668857+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: cluster 2024-02-10T08:33:30.669096+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: audit 2024-02-10T08:33:30.726171+0000 mon.c (mon.2) 735 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: audit 2024-02-10T08:33:30.726828+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: audit 2024-02-10T08:33:30.727825+0000 mon.c (mon.2) 736 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-55"}]: dispatch 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: audit 2024-02-10T08:33:30.728294+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-55"}]: dispatch 2024-02-10T08:33:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:31 smithi134 bash[18722]: audit 2024-02-10T08:33:30.780609+0000 mon.a (mon.0) 2200 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: cluster 2024-02-10T08:33:30.059248+0000 mgr.y (mgr.14547) 439 : cluster [DBG] pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: audit 2024-02-10T08:33:30.668857+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: cluster 2024-02-10T08:33:30.669096+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-10T08:33:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: audit 2024-02-10T08:33:30.726171+0000 mon.c (mon.2) 735 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: audit 2024-02-10T08:33:30.726828+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: audit 2024-02-10T08:33:30.727825+0000 mon.c (mon.2) 736 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-55"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: audit 2024-02-10T08:33:30.728294+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-55"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[17385]: audit 2024-02-10T08:33:30.780609+0000 mon.a (mon.0) 2200 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: cluster 2024-02-10T08:33:30.059248+0000 mgr.y (mgr.14547) 439 : cluster [DBG] pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: audit 2024-02-10T08:33:30.668857+0000 mon.a (mon.0) 2196 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: cluster 2024-02-10T08:33:30.669096+0000 mon.a (mon.0) 2197 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: audit 2024-02-10T08:33:30.726171+0000 mon.c (mon.2) 735 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: audit 2024-02-10T08:33:30.726828+0000 mon.a (mon.0) 2198 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: audit 2024-02-10T08:33:30.727825+0000 mon.c (mon.2) 736 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-55"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: audit 2024-02-10T08:33:30.728294+0000 mon.a (mon.0) 2199 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-55"}]: dispatch 2024-02-10T08:33:32.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:31 smithi067 bash[24488]: audit 2024-02-10T08:33:30.780609+0000 mon.a (mon.0) 2200 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:32.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:33:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:33:33.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:32 smithi134 bash[18722]: cluster 2024-02-10T08:33:31.683526+0000 mon.a (mon.0) 2201 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-10T08:33:33.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:32 smithi067 bash[17385]: cluster 2024-02-10T08:33:31.683526+0000 mon.a (mon.0) 2201 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-10T08:33:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:32 smithi067 bash[24488]: cluster 2024-02-10T08:33:31.683526+0000 mon.a (mon.0) 2201 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-10T08:33:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:33:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:33:34.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:33 smithi134 bash[18722]: cluster 2024-02-10T08:33:32.060033+0000 mgr.y (mgr.14547) 440 : cluster [DBG] pgmap v623: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:33:34.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:33 smithi134 bash[18722]: audit 2024-02-10T08:33:32.102881+0000 mgr.y (mgr.14547) 441 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:34.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:33 smithi134 bash[18722]: cluster 2024-02-10T08:33:32.704662+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-10T08:33:34.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:33 smithi134 bash[18722]: audit 2024-02-10T08:33:32.711413+0000 mon.c (mon.2) 737 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:34.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:33 smithi134 bash[18722]: audit 2024-02-10T08:33:32.712141+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[17385]: cluster 2024-02-10T08:33:32.060033+0000 mgr.y (mgr.14547) 440 : cluster [DBG] pgmap v623: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:33:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[17385]: audit 2024-02-10T08:33:32.102881+0000 mgr.y (mgr.14547) 441 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[17385]: cluster 2024-02-10T08:33:32.704662+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-10T08:33:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[17385]: audit 2024-02-10T08:33:32.711413+0000 mon.c (mon.2) 737 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[17385]: audit 2024-02-10T08:33:32.712141+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[24488]: cluster 2024-02-10T08:33:32.060033+0000 mgr.y (mgr.14547) 440 : cluster [DBG] pgmap v623: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:33:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[24488]: audit 2024-02-10T08:33:32.102881+0000 mgr.y (mgr.14547) 441 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[24488]: cluster 2024-02-10T08:33:32.704662+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-10T08:33:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[24488]: audit 2024-02-10T08:33:32.711413+0000 mon.c (mon.2) 737 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:34.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:33 smithi067 bash[24488]: audit 2024-02-10T08:33:32.712141+0000 mon.a (mon.0) 2203 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: audit 2024-02-10T08:33:33.709137+0000 mon.a (mon.0) 2204 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: cluster 2024-02-10T08:33:33.709526+0000 mon.a (mon.0) 2205 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: audit 2024-02-10T08:33:33.727981+0000 mon.c (mon.2) 738 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: audit 2024-02-10T08:33:33.776840+0000 mon.c (mon.2) 739 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: audit 2024-02-10T08:33:33.777456+0000 mon.a (mon.0) 2206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: audit 2024-02-10T08:33:33.778568+0000 mon.c (mon.2) 740 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-57"}]: dispatch 2024-02-10T08:33:35.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:34 smithi134 bash[18722]: audit 2024-02-10T08:33:33.779283+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-57"}]: dispatch 2024-02-10T08:33:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: audit 2024-02-10T08:33:33.709137+0000 mon.a (mon.0) 2204 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: cluster 2024-02-10T08:33:33.709526+0000 mon.a (mon.0) 2205 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-10T08:33:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: audit 2024-02-10T08:33:33.727981+0000 mon.c (mon.2) 738 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: audit 2024-02-10T08:33:33.776840+0000 mon.c (mon.2) 739 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: audit 2024-02-10T08:33:33.777456+0000 mon.a (mon.0) 2206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: audit 2024-02-10T08:33:33.778568+0000 mon.c (mon.2) 740 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-57"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[24488]: audit 2024-02-10T08:33:33.779283+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-57"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: audit 2024-02-10T08:33:33.709137+0000 mon.a (mon.0) 2204 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: cluster 2024-02-10T08:33:33.709526+0000 mon.a (mon.0) 2205 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: audit 2024-02-10T08:33:33.727981+0000 mon.c (mon.2) 738 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: audit 2024-02-10T08:33:33.776840+0000 mon.c (mon.2) 739 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: audit 2024-02-10T08:33:33.777456+0000 mon.a (mon.0) 2206 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: audit 2024-02-10T08:33:33.778568+0000 mon.c (mon.2) 740 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-57"}]: dispatch 2024-02-10T08:33:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:34 smithi067 bash[17385]: audit 2024-02-10T08:33:33.779283+0000 mon.a (mon.0) 2207 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-57"}]: dispatch 2024-02-10T08:33:36.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:35 smithi134 bash[18722]: cluster 2024-02-10T08:33:34.060781+0000 mgr.y (mgr.14547) 442 : cluster [DBG] pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:33:36.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:35 smithi134 bash[18722]: cluster 2024-02-10T08:33:34.730837+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-10T08:33:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:35 smithi067 bash[17385]: cluster 2024-02-10T08:33:34.060781+0000 mgr.y (mgr.14547) 442 : cluster [DBG] pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:33:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:35 smithi067 bash[17385]: cluster 2024-02-10T08:33:34.730837+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-10T08:33:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:35 smithi067 bash[24488]: cluster 2024-02-10T08:33:34.060781+0000 mgr.y (mgr.14547) 442 : cluster [DBG] pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:33:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:35 smithi067 bash[24488]: cluster 2024-02-10T08:33:34.730837+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-10T08:33:37.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:36 smithi134 bash[18722]: cluster 2024-02-10T08:33:35.749788+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-10T08:33:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:36 smithi134 bash[18722]: audit 2024-02-10T08:33:35.759014+0000 mon.c (mon.2) 741 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:37.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:36 smithi134 bash[18722]: audit 2024-02-10T08:33:35.759826+0000 mon.a (mon.0) 2210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:36 smithi067 bash[17385]: cluster 2024-02-10T08:33:35.749788+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-10T08:33:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:36 smithi067 bash[17385]: audit 2024-02-10T08:33:35.759014+0000 mon.c (mon.2) 741 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:36 smithi067 bash[17385]: audit 2024-02-10T08:33:35.759826+0000 mon.a (mon.0) 2210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:36 smithi067 bash[24488]: cluster 2024-02-10T08:33:35.749788+0000 mon.a (mon.0) 2209 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-10T08:33:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:36 smithi067 bash[24488]: audit 2024-02-10T08:33:35.759014+0000 mon.c (mon.2) 741 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:37.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:36 smithi067 bash[24488]: audit 2024-02-10T08:33:35.759826+0000 mon.a (mon.0) 2210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:37 smithi134 bash[18722]: cluster 2024-02-10T08:33:36.061710+0000 mgr.y (mgr.14547) 443 : cluster [DBG] pgmap v629: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:37 smithi134 bash[18722]: audit 2024-02-10T08:33:36.760633+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:37 smithi134 bash[18722]: cluster 2024-02-10T08:33:36.760806+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-10T08:33:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:37 smithi134 bash[18722]: audit 2024-02-10T08:33:36.765772+0000 mon.c (mon.2) 742 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:37 smithi134 bash[18722]: audit 2024-02-10T08:33:36.776040+0000 mon.c (mon.2) 743 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:38.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:37 smithi134 bash[18722]: audit 2024-02-10T08:33:36.776724+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:37] "GET /metrics HTTP/1.1" 200 188313 "" "Prometheus/2.33.4" 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17385]: cluster 2024-02-10T08:33:36.061710+0000 mgr.y (mgr.14547) 443 : cluster [DBG] pgmap v629: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17385]: audit 2024-02-10T08:33:36.760633+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17385]: cluster 2024-02-10T08:33:36.760806+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17385]: audit 2024-02-10T08:33:36.765772+0000 mon.c (mon.2) 742 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17385]: audit 2024-02-10T08:33:36.776040+0000 mon.c (mon.2) 743 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[17385]: audit 2024-02-10T08:33:36.776724+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[24488]: cluster 2024-02-10T08:33:36.061710+0000 mgr.y (mgr.14547) 443 : cluster [DBG] pgmap v629: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[24488]: audit 2024-02-10T08:33:36.760633+0000 mon.a (mon.0) 2211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[24488]: cluster 2024-02-10T08:33:36.760806+0000 mon.a (mon.0) 2212 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-10T08:33:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[24488]: audit 2024-02-10T08:33:36.765772+0000 mon.c (mon.2) 742 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[24488]: audit 2024-02-10T08:33:36.776040+0000 mon.c (mon.2) 743 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:38.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:37 smithi067 bash[24488]: audit 2024-02-10T08:33:36.776724+0000 mon.a (mon.0) 2213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:39.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:38 smithi134 bash[18722]: audit 2024-02-10T08:33:37.764399+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:33:39.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:38 smithi134 bash[18722]: cluster 2024-02-10T08:33:37.764617+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-10T08:33:39.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:38 smithi134 bash[18722]: audit 2024-02-10T08:33:37.825264+0000 mon.c (mon.2) 744 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:39.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:38 smithi134 bash[18722]: audit 2024-02-10T08:33:37.825919+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:39.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:38 smithi134 bash[18722]: audit 2024-02-10T08:33:37.826832+0000 mon.c (mon.2) 745 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-59"}]: dispatch 2024-02-10T08:33:39.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:38 smithi134 bash[18722]: audit 2024-02-10T08:33:37.827362+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-59"}]: dispatch 2024-02-10T08:33:39.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[17385]: audit 2024-02-10T08:33:37.764399+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:33:39.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[17385]: cluster 2024-02-10T08:33:37.764617+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[17385]: audit 2024-02-10T08:33:37.825264+0000 mon.c (mon.2) 744 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[17385]: audit 2024-02-10T08:33:37.825919+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[17385]: audit 2024-02-10T08:33:37.826832+0000 mon.c (mon.2) 745 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-59"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[17385]: audit 2024-02-10T08:33:37.827362+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-59"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[24488]: audit 2024-02-10T08:33:37.764399+0000 mon.a (mon.0) 2214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[24488]: cluster 2024-02-10T08:33:37.764617+0000 mon.a (mon.0) 2215 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[24488]: audit 2024-02-10T08:33:37.825264+0000 mon.c (mon.2) 744 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[24488]: audit 2024-02-10T08:33:37.825919+0000 mon.a (mon.0) 2216 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[24488]: audit 2024-02-10T08:33:37.826832+0000 mon.c (mon.2) 745 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-59"}]: dispatch 2024-02-10T08:33:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:38 smithi067 bash[24488]: audit 2024-02-10T08:33:37.827362+0000 mon.a (mon.0) 2217 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-59"}]: dispatch 2024-02-10T08:33:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:39 smithi067 bash[17385]: cluster 2024-02-10T08:33:38.062580+0000 mgr.y (mgr.14547) 444 : cluster [DBG] pgmap v632: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:33:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:39 smithi067 bash[17385]: cluster 2024-02-10T08:33:38.787369+0000 mon.a (mon.0) 2218 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-10T08:33:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:39 smithi067 bash[24488]: cluster 2024-02-10T08:33:38.062580+0000 mgr.y (mgr.14547) 444 : cluster [DBG] pgmap v632: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:33:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:39 smithi067 bash[24488]: cluster 2024-02-10T08:33:38.787369+0000 mon.a (mon.0) 2218 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-10T08:33:40.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:39 smithi134 bash[18722]: cluster 2024-02-10T08:33:38.062580+0000 mgr.y (mgr.14547) 444 : cluster [DBG] pgmap v632: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:33:40.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:39 smithi134 bash[18722]: cluster 2024-02-10T08:33:38.787369+0000 mon.a (mon.0) 2218 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-10T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:40 smithi067 bash[17385]: cluster 2024-02-10T08:33:39.803264+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-10T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:40 smithi067 bash[17385]: audit 2024-02-10T08:33:39.813824+0000 mon.c (mon.2) 746 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:40 smithi067 bash[17385]: audit 2024-02-10T08:33:39.814396+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:40 smithi067 bash[24488]: cluster 2024-02-10T08:33:39.803264+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-10T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:40 smithi067 bash[24488]: audit 2024-02-10T08:33:39.813824+0000 mon.c (mon.2) 746 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:41.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:40 smithi067 bash[24488]: audit 2024-02-10T08:33:39.814396+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:40 smithi134 bash[18722]: cluster 2024-02-10T08:33:39.803264+0000 mon.a (mon.0) 2219 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-10T08:33:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:40 smithi134 bash[18722]: audit 2024-02-10T08:33:39.813824+0000 mon.c (mon.2) 746 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:40 smithi134 bash[18722]: audit 2024-02-10T08:33:39.814396+0000 mon.a (mon.0) 2220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:33:42.116 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: cluster 2024-02-10T08:33:40.063369+0000 mgr.y (mgr.14547) 445 : cluster [DBG] pgmap v635: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:33:42.116 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:40.807702+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:42.116 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: cluster 2024-02-10T08:33:40.808343+0000 mon.a (mon.0) 2222 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-10T08:33:42.116 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:40.811132+0000 mon.c (mon.2) 747 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:42.116 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:40.815507+0000 mon.c (mon.2) 748 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:42.116 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:40.827711+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.397318+0000 mon.c (mon.2) 749 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.697705+0000 mon.c (mon.2) 750 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.698516+0000 mon.a (mon.0) 2224 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.700521+0000 mon.c (mon.2) 751 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.701122+0000 mon.a (mon.0) 2225 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.702748+0000 mon.c (mon.2) 752 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.703255+0000 mon.a (mon.0) 2226 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.704664+0000 mon.c (mon.2) 753 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.705161+0000 mon.a (mon.0) 2227 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.708391+0000 mon.c (mon.2) 754 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.709024+0000 mon.a (mon.0) 2228 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.713723+0000 mon.a (mon.0) 2229 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.715603+0000 mon.c (mon.2) 755 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.117 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.716064+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.717234+0000 mon.c (mon.2) 756 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.717692+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.718981+0000 mon.c (mon.2) 757 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.719467+0000 mon.a (mon.0) 2232 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.730479+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.732524+0000 mon.c (mon.2) 758 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.733886+0000 mon.c (mon.2) 759 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:33:42.118 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:41 smithi134 bash[18722]: audit 2024-02-10T08:33:41.743369+0000 mon.a (mon.0) 2234 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: cluster 2024-02-10T08:33:40.063369+0000 mgr.y (mgr.14547) 445 : cluster [DBG] pgmap v635: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:33:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:40.807702+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: cluster 2024-02-10T08:33:40.808343+0000 mon.a (mon.0) 2222 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:40.811132+0000 mon.c (mon.2) 747 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:40.815507+0000 mon.c (mon.2) 748 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:40.827711+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.397318+0000 mon.c (mon.2) 749 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.697705+0000 mon.c (mon.2) 750 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.698516+0000 mon.a (mon.0) 2224 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.700521+0000 mon.c (mon.2) 751 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.701122+0000 mon.a (mon.0) 2225 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.702748+0000 mon.c (mon.2) 752 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.703255+0000 mon.a (mon.0) 2226 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.704664+0000 mon.c (mon.2) 753 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.705161+0000 mon.a (mon.0) 2227 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.708391+0000 mon.c (mon.2) 754 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: cluster 2024-02-10T08:33:40.063369+0000 mgr.y (mgr.14547) 445 : cluster [DBG] pgmap v635: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-10T08:33:42.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:40.807702+0000 mon.a (mon.0) 2221 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:33:42.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: cluster 2024-02-10T08:33:40.808343+0000 mon.a (mon.0) 2222 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-10T08:33:42.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:40.811132+0000 mon.c (mon.2) 747 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:33:42.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:40.815507+0000 mon.c (mon.2) 748 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:42.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:40.827711+0000 mon.a (mon.0) 2223 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.397318+0000 mon.c (mon.2) 749 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.697705+0000 mon.c (mon.2) 750 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.698516+0000 mon.a (mon.0) 2224 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.700521+0000 mon.c (mon.2) 751 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.701122+0000 mon.a (mon.0) 2225 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.702748+0000 mon.c (mon.2) 752 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.703255+0000 mon.a (mon.0) 2226 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.704664+0000 mon.c (mon.2) 753 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.705161+0000 mon.a (mon.0) 2227 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.708391+0000 mon.c (mon.2) 754 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.709024+0000 mon.a (mon.0) 2228 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.713723+0000 mon.a (mon.0) 2229 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.715603+0000 mon.c (mon.2) 755 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.716064+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.717234+0000 mon.c (mon.2) 756 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.717692+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.718981+0000 mon.c (mon.2) 757 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.719467+0000 mon.a (mon.0) 2232 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.730479+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.732524+0000 mon.c (mon.2) 758 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.733886+0000 mon.c (mon.2) 759 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[24488]: audit 2024-02-10T08:33:41.743369+0000 mon.a (mon.0) 2234 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.709024+0000 mon.a (mon.0) 2228 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.713723+0000 mon.a (mon.0) 2229 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.715603+0000 mon.c (mon.2) 755 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.716064+0000 mon.a (mon.0) 2230 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.717234+0000 mon.c (mon.2) 756 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.717692+0000 mon.a (mon.0) 2231 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.718981+0000 mon.c (mon.2) 757 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.719467+0000 mon.a (mon.0) 2232 : audit [INF] from='mgr.14547 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.730479+0000 mon.a (mon.0) 2233 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.732524+0000 mon.c (mon.2) 758 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.733886+0000 mon.c (mon.2) 759 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:33:42.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:41 smithi067 bash[17385]: audit 2024-02-10T08:33:41.743369+0000 mon.a (mon.0) 2234 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:33:42.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:33:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[17385]: cephadm 2024-02-10T08:33:41.707220+0000 mgr.y (mgr.14547) 446 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 3292M 2024-02-10T08:33:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[17385]: cephadm 2024-02-10T08:33:41.720454+0000 mgr.y (mgr.14547) 447 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 3292M 2024-02-10T08:33:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[17385]: audit 2024-02-10T08:33:41.838107+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[17385]: cluster 2024-02-10T08:33:41.838549+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[17385]: cluster 2024-02-10T08:33:42.064311+0000 mgr.y (mgr.14547) 448 : cluster [DBG] pgmap v638: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[17385]: audit 2024-02-10T08:33:42.114019+0000 mgr.y (mgr.14547) 449 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[24488]: cephadm 2024-02-10T08:33:41.707220+0000 mgr.y (mgr.14547) 446 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 3292M 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[24488]: cephadm 2024-02-10T08:33:41.720454+0000 mgr.y (mgr.14547) 447 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 3292M 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[24488]: audit 2024-02-10T08:33:41.838107+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[24488]: cluster 2024-02-10T08:33:41.838549+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[24488]: cluster 2024-02-10T08:33:42.064311+0000 mgr.y (mgr.14547) 448 : cluster [DBG] pgmap v638: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:42 smithi067 bash[24488]: audit 2024-02-10T08:33:42.114019+0000 mgr.y (mgr.14547) 449 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[18722]: cephadm 2024-02-10T08:33:41.707220+0000 mgr.y (mgr.14547) 446 : cephadm [INF] Adjusting osd_memory_target on smithi134 to 3292M 2024-02-10T08:33:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[18722]: cephadm 2024-02-10T08:33:41.720454+0000 mgr.y (mgr.14547) 447 : cephadm [INF] Adjusting osd_memory_target on smithi067 to 3292M 2024-02-10T08:33:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[18722]: audit 2024-02-10T08:33:41.838107+0000 mon.a (mon.0) 2235 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:33:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[18722]: cluster 2024-02-10T08:33:41.838549+0000 mon.a (mon.0) 2236 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-10T08:33:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[18722]: cluster 2024-02-10T08:33:42.064311+0000 mgr.y (mgr.14547) 448 : cluster [DBG] pgmap v638: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:42 smithi134 bash[18722]: audit 2024-02-10T08:33:42.114019+0000 mgr.y (mgr.14547) 449 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:33:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:33:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:43 smithi067 bash[17385]: cluster 2024-02-10T08:33:42.840180+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-10T08:33:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:43 smithi067 bash[24488]: cluster 2024-02-10T08:33:42.840180+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-10T08:33:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:43 smithi134 bash[18722]: cluster 2024-02-10T08:33:42.840180+0000 mon.a (mon.0) 2237 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-10T08:33:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:44 smithi067 bash[17385]: cluster 2024-02-10T08:33:43.850846+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-10T08:33:45.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:44 smithi067 bash[17385]: cluster 2024-02-10T08:33:44.065149+0000 mgr.y (mgr.14547) 450 : cluster [DBG] pgmap v641: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:45.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:44 smithi067 bash[24488]: cluster 2024-02-10T08:33:43.850846+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-10T08:33:45.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:44 smithi067 bash[24488]: cluster 2024-02-10T08:33:44.065149+0000 mgr.y (mgr.14547) 450 : cluster [DBG] pgmap v641: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:45.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:44 smithi134 bash[18722]: cluster 2024-02-10T08:33:43.850846+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-10T08:33:45.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:44 smithi134 bash[18722]: cluster 2024-02-10T08:33:44.065149+0000 mgr.y (mgr.14547) 450 : cluster [DBG] pgmap v641: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:33:46.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:45 smithi134 bash[18722]: cluster 2024-02-10T08:33:44.855450+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-10T08:33:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:45 smithi067 bash[17385]: cluster 2024-02-10T08:33:44.855450+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-10T08:33:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:45 smithi067 bash[24488]: cluster 2024-02-10T08:33:44.855450+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-10T08:33:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:46 smithi134 bash[18722]: cluster 2024-02-10T08:33:45.870906+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-10T08:33:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:46 smithi134 bash[18722]: cluster 2024-02-10T08:33:46.065966+0000 mgr.y (mgr.14547) 451 : cluster [DBG] pgmap v644: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:46 smithi067 bash[17385]: cluster 2024-02-10T08:33:45.870906+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-10T08:33:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:46 smithi067 bash[17385]: cluster 2024-02-10T08:33:46.065966+0000 mgr.y (mgr.14547) 451 : cluster [DBG] pgmap v644: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:47.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:46 smithi067 bash[24488]: cluster 2024-02-10T08:33:45.870906+0000 mon.a (mon.0) 2240 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-10T08:33:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:46 smithi067 bash[24488]: cluster 2024-02-10T08:33:46.065966+0000 mgr.y (mgr.14547) 451 : cluster [DBG] pgmap v644: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:33:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:33:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:47] "GET /metrics HTTP/1.1" 200 188457 "" "Prometheus/2.33.4" 2024-02-10T08:33:49.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:49 smithi134 bash[18722]: cluster 2024-02-10T08:33:48.068113+0000 mgr.y (mgr.14547) 452 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-02-10T08:33:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:49 smithi067 bash[17385]: cluster 2024-02-10T08:33:48.068113+0000 mgr.y (mgr.14547) 452 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-02-10T08:33:49.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:49 smithi067 bash[24488]: cluster 2024-02-10T08:33:48.068113+0000 mgr.y (mgr.14547) 452 : cluster [DBG] pgmap v645: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-02-10T08:33:51.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:51 smithi134 bash[18722]: cluster 2024-02-10T08:33:50.068812+0000 mgr.y (mgr.14547) 453 : cluster [DBG] pgmap v646: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-02-10T08:33:51.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:51 smithi134 bash[18722]: cluster 2024-02-10T08:33:51.115203+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-10T08:33:51.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:51 smithi067 bash[24488]: cluster 2024-02-10T08:33:50.068812+0000 mgr.y (mgr.14547) 453 : cluster [DBG] pgmap v646: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-02-10T08:33:51.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:51 smithi067 bash[24488]: cluster 2024-02-10T08:33:51.115203+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-10T08:33:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:51 smithi067 bash[17385]: cluster 2024-02-10T08:33:50.068812+0000 mgr.y (mgr.14547) 453 : cluster [DBG] pgmap v646: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-02-10T08:33:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:51 smithi067 bash[17385]: cluster 2024-02-10T08:33:51.115203+0000 mon.a (mon.0) 2241 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-10T08:33:52.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:33:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:33:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:33:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:33:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:53 smithi134 bash[18722]: cluster 2024-02-10T08:33:52.070952+0000 mgr.y (mgr.14547) 454 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T08:33:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:53 smithi134 bash[18722]: audit 2024-02-10T08:33:52.123068+0000 mgr.y (mgr.14547) 455 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:53 smithi067 bash[17385]: cluster 2024-02-10T08:33:52.070952+0000 mgr.y (mgr.14547) 454 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T08:33:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:53 smithi067 bash[17385]: audit 2024-02-10T08:33:52.123068+0000 mgr.y (mgr.14547) 455 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:53 smithi067 bash[24488]: cluster 2024-02-10T08:33:52.070952+0000 mgr.y (mgr.14547) 454 : cluster [DBG] pgmap v648: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T08:33:53.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:53 smithi067 bash[24488]: audit 2024-02-10T08:33:52.123068+0000 mgr.y (mgr.14547) 455 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:33:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:55 smithi134 bash[18722]: cluster 2024-02-10T08:33:54.071698+0000 mgr.y (mgr.14547) 456 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T08:33:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:55 smithi067 bash[17385]: cluster 2024-02-10T08:33:54.071698+0000 mgr.y (mgr.14547) 456 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T08:33:55.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:55 smithi067 bash[24488]: cluster 2024-02-10T08:33:54.071698+0000 mgr.y (mgr.14547) 456 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-10T08:33:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:57 smithi134 bash[18722]: cluster 2024-02-10T08:33:56.072619+0000 mgr.y (mgr.14547) 457 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-10T08:33:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:57 smithi134 bash[18722]: cluster 2024-02-10T08:33:56.152621+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-10T08:33:57.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:57 smithi067 bash[17385]: cluster 2024-02-10T08:33:56.072619+0000 mgr.y (mgr.14547) 457 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-10T08:33:57.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:57 smithi067 bash[17385]: cluster 2024-02-10T08:33:56.152621+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-10T08:33:57.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:57 smithi067 bash[24488]: cluster 2024-02-10T08:33:56.072619+0000 mgr.y (mgr.14547) 457 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-02-10T08:33:57.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:57 smithi067 bash[24488]: cluster 2024-02-10T08:33:56.152621+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-10T08:33:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:33:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:33:57] "GET /metrics HTTP/1.1" 200 188457 "" "Prometheus/2.33.4" 2024-02-10T08:33:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:33:59 smithi134 bash[18722]: cluster 2024-02-10T08:33:58.074542+0000 mgr.y (mgr.14547) 458 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:33:59 smithi067 bash[17385]: cluster 2024-02-10T08:33:58.074542+0000 mgr.y (mgr.14547) 458 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:33:59.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:33:59 smithi067 bash[24488]: cluster 2024-02-10T08:33:58.074542+0000 mgr.y (mgr.14547) 458 : cluster [DBG] pgmap v652: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:01 smithi134 bash[18722]: cluster 2024-02-10T08:34:00.075356+0000 mgr.y (mgr.14547) 459 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:01 smithi134 bash[18722]: cluster 2024-02-10T08:34:01.118165+0000 mon.a (mon.0) 2243 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-10T08:34:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:01 smithi067 bash[17385]: cluster 2024-02-10T08:34:00.075356+0000 mgr.y (mgr.14547) 459 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:01 smithi067 bash[17385]: cluster 2024-02-10T08:34:01.118165+0000 mon.a (mon.0) 2243 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-10T08:34:01.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:01 smithi067 bash[24488]: cluster 2024-02-10T08:34:00.075356+0000 mgr.y (mgr.14547) 459 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:01.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:01 smithi067 bash[24488]: cluster 2024-02-10T08:34:01.118165+0000 mon.a (mon.0) 2243 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-10T08:34:02.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:34:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:34:03.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:34:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:34:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:03 smithi134 bash[18722]: cluster 2024-02-10T08:34:02.077323+0000 mgr.y (mgr.14547) 460 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:03 smithi134 bash[18722]: audit 2024-02-10T08:34:02.131148+0000 mgr.y (mgr.14547) 461 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:03 smithi067 bash[17385]: cluster 2024-02-10T08:34:02.077323+0000 mgr.y (mgr.14547) 460 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:03 smithi067 bash[17385]: audit 2024-02-10T08:34:02.131148+0000 mgr.y (mgr.14547) 461 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:03 smithi067 bash[24488]: cluster 2024-02-10T08:34:02.077323+0000 mgr.y (mgr.14547) 460 : cluster [DBG] pgmap v655: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:03 smithi067 bash[24488]: audit 2024-02-10T08:34:02.131148+0000 mgr.y (mgr.14547) 461 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:05.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:05 smithi134 bash[18722]: cluster 2024-02-10T08:34:04.078116+0000 mgr.y (mgr.14547) 462 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:05 smithi067 bash[17385]: cluster 2024-02-10T08:34:04.078116+0000 mgr.y (mgr.14547) 462 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:05 smithi067 bash[24488]: cluster 2024-02-10T08:34:04.078116+0000 mgr.y (mgr.14547) 462 : cluster [DBG] pgmap v656: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:07.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:07 smithi134 bash[18722]: cluster 2024-02-10T08:34:06.079010+0000 mgr.y (mgr.14547) 463 : cluster [DBG] pgmap v657: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:07.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:07 smithi134 bash[18722]: cluster 2024-02-10T08:34:07.123006+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-10T08:34:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:07 smithi067 bash[24488]: cluster 2024-02-10T08:34:06.079010+0000 mgr.y (mgr.14547) 463 : cluster [DBG] pgmap v657: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:07 smithi067 bash[24488]: cluster 2024-02-10T08:34:07.123006+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-10T08:34:07.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:07 smithi067 bash[17385]: cluster 2024-02-10T08:34:06.079010+0000 mgr.y (mgr.14547) 463 : cluster [DBG] pgmap v657: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:34:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:07 smithi067 bash[17385]: cluster 2024-02-10T08:34:07.123006+0000 mon.a (mon.0) 2244 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-10T08:34:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:34:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:07] "GET /metrics HTTP/1.1" 200 188453 "" "Prometheus/2.33.4" 2024-02-10T08:34:09.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:09 smithi134 bash[18722]: cluster 2024-02-10T08:34:08.079790+0000 mgr.y (mgr.14547) 464 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:34:09.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:09 smithi067 bash[17385]: cluster 2024-02-10T08:34:08.079790+0000 mgr.y (mgr.14547) 464 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:34:09.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:09 smithi067 bash[24488]: cluster 2024-02-10T08:34:08.079790+0000 mgr.y (mgr.14547) 464 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:34:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:11 smithi134 bash[18722]: cluster 2024-02-10T08:34:10.080646+0000 mgr.y (mgr.14547) 465 : cluster [DBG] pgmap v660: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:34:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:11 smithi067 bash[17385]: cluster 2024-02-10T08:34:10.080646+0000 mgr.y (mgr.14547) 465 : cluster [DBG] pgmap v660: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:34:11.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:11 smithi067 bash[24488]: cluster 2024-02-10T08:34:10.080646+0000 mgr.y (mgr.14547) 465 : cluster [DBG] pgmap v660: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:34:12.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:34:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:34:13.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:34:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:34:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:13 smithi134 bash[18722]: cluster 2024-02-10T08:34:12.082963+0000 mgr.y (mgr.14547) 466 : cluster [DBG] pgmap v661: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:13 smithi134 bash[18722]: audit 2024-02-10T08:34:12.138897+0000 mgr.y (mgr.14547) 467 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:13 smithi067 bash[17385]: cluster 2024-02-10T08:34:12.082963+0000 mgr.y (mgr.14547) 466 : cluster [DBG] pgmap v661: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:13 smithi067 bash[17385]: audit 2024-02-10T08:34:12.138897+0000 mgr.y (mgr.14547) 467 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:13 smithi067 bash[24488]: cluster 2024-02-10T08:34:12.082963+0000 mgr.y (mgr.14547) 466 : cluster [DBG] pgmap v661: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:13 smithi067 bash[24488]: audit 2024-02-10T08:34:12.138897+0000 mgr.y (mgr.14547) 467 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:15 smithi134 bash[18722]: cluster 2024-02-10T08:34:14.083737+0000 mgr.y (mgr.14547) 468 : cluster [DBG] pgmap v662: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:15 smithi067 bash[17385]: cluster 2024-02-10T08:34:14.083737+0000 mgr.y (mgr.14547) 468 : cluster [DBG] pgmap v662: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:15.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:15 smithi067 bash[24488]: cluster 2024-02-10T08:34:14.083737+0000 mgr.y (mgr.14547) 468 : cluster [DBG] pgmap v662: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:17.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:17 smithi134 bash[18722]: cluster 2024-02-10T08:34:16.084560+0000 mgr.y (mgr.14547) 469 : cluster [DBG] pgmap v663: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:17.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:17 smithi134 bash[18722]: cluster 2024-02-10T08:34:16.123174+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-10T08:34:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:17 smithi067 bash[17385]: cluster 2024-02-10T08:34:16.084560+0000 mgr.y (mgr.14547) 469 : cluster [DBG] pgmap v663: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:17 smithi067 bash[17385]: cluster 2024-02-10T08:34:16.123174+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-10T08:34:17.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:17 smithi067 bash[24488]: cluster 2024-02-10T08:34:16.084560+0000 mgr.y (mgr.14547) 469 : cluster [DBG] pgmap v663: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:34:17.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:17 smithi067 bash[24488]: cluster 2024-02-10T08:34:16.123174+0000 mon.a (mon.0) 2245 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-10T08:34:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:34:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:17] "GET /metrics HTTP/1.1" 200 188440 "" "Prometheus/2.33.4" 2024-02-10T08:34:18.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:18 smithi134 bash[18722]: audit 2024-02-10T08:34:17.159310+0000 mon.c (mon.2) 760 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:18.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:18 smithi134 bash[18722]: audit 2024-02-10T08:34:17.159902+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:18 smithi134 bash[18722]: audit 2024-02-10T08:34:17.160846+0000 mon.c (mon.2) 761 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-61"}]: dispatch 2024-02-10T08:34:18.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:18 smithi134 bash[18722]: audit 2024-02-10T08:34:17.161416+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-61"}]: dispatch 2024-02-10T08:34:18.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[24488]: audit 2024-02-10T08:34:17.159310+0000 mon.c (mon.2) 760 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:18.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[24488]: audit 2024-02-10T08:34:17.159902+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:18.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[24488]: audit 2024-02-10T08:34:17.160846+0000 mon.c (mon.2) 761 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-61"}]: dispatch 2024-02-10T08:34:18.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[24488]: audit 2024-02-10T08:34:17.161416+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-61"}]: dispatch 2024-02-10T08:34:18.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[17385]: audit 2024-02-10T08:34:17.159310+0000 mon.c (mon.2) 760 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:18.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[17385]: audit 2024-02-10T08:34:17.159902+0000 mon.a (mon.0) 2246 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:18.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[17385]: audit 2024-02-10T08:34:17.160846+0000 mon.c (mon.2) 761 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-61"}]: dispatch 2024-02-10T08:34:18.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:18 smithi067 bash[17385]: audit 2024-02-10T08:34:17.161416+0000 mon.a (mon.0) 2247 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-61"}]: dispatch 2024-02-10T08:34:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:19 smithi134 bash[18722]: cluster 2024-02-10T08:34:18.086739+0000 mgr.y (mgr.14547) 470 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:34:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:19 smithi134 bash[18722]: cluster 2024-02-10T08:34:18.135032+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-10T08:34:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:19 smithi067 bash[17385]: cluster 2024-02-10T08:34:18.086739+0000 mgr.y (mgr.14547) 470 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:34:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:19 smithi067 bash[17385]: cluster 2024-02-10T08:34:18.135032+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-10T08:34:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:19 smithi067 bash[24488]: cluster 2024-02-10T08:34:18.086739+0000 mgr.y (mgr.14547) 470 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:34:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:19 smithi067 bash[24488]: cluster 2024-02-10T08:34:18.135032+0000 mon.a (mon.0) 2248 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-10T08:34:20.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:20 smithi134 bash[18722]: cluster 2024-02-10T08:34:19.148801+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-10T08:34:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:20 smithi134 bash[18722]: audit 2024-02-10T08:34:19.151238+0000 mon.c (mon.2) 762 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:20.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:20 smithi134 bash[18722]: audit 2024-02-10T08:34:19.153052+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:20 smithi067 bash[17385]: cluster 2024-02-10T08:34:19.148801+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-10T08:34:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:20 smithi067 bash[17385]: audit 2024-02-10T08:34:19.151238+0000 mon.c (mon.2) 762 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:20 smithi067 bash[17385]: audit 2024-02-10T08:34:19.153052+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:20.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:20 smithi067 bash[24488]: cluster 2024-02-10T08:34:19.148801+0000 mon.a (mon.0) 2249 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-10T08:34:20.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:20 smithi067 bash[24488]: audit 2024-02-10T08:34:19.151238+0000 mon.c (mon.2) 762 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:20.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:20 smithi067 bash[24488]: audit 2024-02-10T08:34:19.153052+0000 mon.a (mon.0) 2250 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:21.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: cluster 2024-02-10T08:34:20.087526+0000 mgr.y (mgr.14547) 471 : cluster [DBG] pgmap v668: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: audit 2024-02-10T08:34:20.152664+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: cluster 2024-02-10T08:34:20.153165+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: audit 2024-02-10T08:34:20.176302+0000 mon.c (mon.2) 763 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: audit 2024-02-10T08:34:20.182346+0000 mon.c (mon.2) 764 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: audit 2024-02-10T08:34:20.182717+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: audit 2024-02-10T08:34:21.154948+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:34:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:21 smithi134 bash[18722]: cluster 2024-02-10T08:34:21.155093+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-10T08:34:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: cluster 2024-02-10T08:34:20.087526+0000 mgr.y (mgr.14547) 471 : cluster [DBG] pgmap v668: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: audit 2024-02-10T08:34:20.152664+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: cluster 2024-02-10T08:34:20.153165+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-10T08:34:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: audit 2024-02-10T08:34:20.176302+0000 mon.c (mon.2) 763 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: audit 2024-02-10T08:34:20.182346+0000 mon.c (mon.2) 764 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: audit 2024-02-10T08:34:20.182717+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: audit 2024-02-10T08:34:21.154948+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[17385]: cluster 2024-02-10T08:34:21.155093+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: cluster 2024-02-10T08:34:20.087526+0000 mgr.y (mgr.14547) 471 : cluster [DBG] pgmap v668: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: audit 2024-02-10T08:34:20.152664+0000 mon.a (mon.0) 2251 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: cluster 2024-02-10T08:34:20.153165+0000 mon.a (mon.0) 2252 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: audit 2024-02-10T08:34:20.176302+0000 mon.c (mon.2) 763 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: audit 2024-02-10T08:34:20.182346+0000 mon.c (mon.2) 764 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: audit 2024-02-10T08:34:20.182717+0000 mon.a (mon.0) 2253 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: audit 2024-02-10T08:34:21.154948+0000 mon.a (mon.0) 2254 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:34:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:21 smithi067 bash[24488]: cluster 2024-02-10T08:34:21.155093+0000 mon.a (mon.0) 2255 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-10T08:34:22.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:34:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:34:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:34:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:34:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:23 smithi134 bash[18722]: cluster 2024-02-10T08:34:22.089722+0000 mgr.y (mgr.14547) 472 : cluster [DBG] pgmap v671: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:23 smithi134 bash[18722]: audit 2024-02-10T08:34:22.148405+0000 mgr.y (mgr.14547) 473 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:23 smithi134 bash[18722]: cluster 2024-02-10T08:34:22.180018+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-10T08:34:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:23 smithi067 bash[17385]: cluster 2024-02-10T08:34:22.089722+0000 mgr.y (mgr.14547) 472 : cluster [DBG] pgmap v671: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:23 smithi067 bash[17385]: audit 2024-02-10T08:34:22.148405+0000 mgr.y (mgr.14547) 473 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:23 smithi067 bash[17385]: cluster 2024-02-10T08:34:22.180018+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-10T08:34:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:23 smithi067 bash[24488]: cluster 2024-02-10T08:34:22.089722+0000 mgr.y (mgr.14547) 472 : cluster [DBG] pgmap v671: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:23 smithi067 bash[24488]: audit 2024-02-10T08:34:22.148405+0000 mgr.y (mgr.14547) 473 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:23 smithi067 bash[24488]: cluster 2024-02-10T08:34:22.180018+0000 mon.a (mon.0) 2256 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-10T08:34:24.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:24 smithi134 bash[18722]: cluster 2024-02-10T08:34:23.185904+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-10T08:34:24.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:24 smithi067 bash[24488]: cluster 2024-02-10T08:34:23.185904+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-10T08:34:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:24 smithi067 bash[17385]: cluster 2024-02-10T08:34:23.185904+0000 mon.a (mon.0) 2257 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-10T08:34:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:25 smithi134 bash[18722]: cluster 2024-02-10T08:34:24.090587+0000 mgr.y (mgr.14547) 474 : cluster [DBG] pgmap v674: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:34:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:25 smithi134 bash[18722]: cluster 2024-02-10T08:34:24.203875+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-10T08:34:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:25 smithi067 bash[17385]: cluster 2024-02-10T08:34:24.090587+0000 mgr.y (mgr.14547) 474 : cluster [DBG] pgmap v674: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:34:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:25 smithi067 bash[17385]: cluster 2024-02-10T08:34:24.203875+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-10T08:34:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:25 smithi067 bash[24488]: cluster 2024-02-10T08:34:24.090587+0000 mgr.y (mgr.14547) 474 : cluster [DBG] pgmap v674: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-10T08:34:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:25 smithi067 bash[24488]: cluster 2024-02-10T08:34:24.203875+0000 mon.a (mon.0) 2258 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-10T08:34:27.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:27 smithi134 bash[18722]: cluster 2024-02-10T08:34:26.091416+0000 mgr.y (mgr.14547) 475 : cluster [DBG] pgmap v676: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T08:34:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:27 smithi067 bash[17385]: cluster 2024-02-10T08:34:26.091416+0000 mgr.y (mgr.14547) 475 : cluster [DBG] pgmap v676: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T08:34:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:27 smithi067 bash[24488]: cluster 2024-02-10T08:34:26.091416+0000 mgr.y (mgr.14547) 475 : cluster [DBG] pgmap v676: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T08:34:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:34:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:27] "GET /metrics HTTP/1.1" 200 188440 "" "Prometheus/2.33.4" 2024-02-10T08:34:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:29 smithi134 bash[18722]: cluster 2024-02-10T08:34:28.093742+0000 mgr.y (mgr.14547) 476 : cluster [DBG] pgmap v677: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-10T08:34:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:29 smithi067 bash[17385]: cluster 2024-02-10T08:34:28.093742+0000 mgr.y (mgr.14547) 476 : cluster [DBG] pgmap v677: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-10T08:34:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:29 smithi067 bash[24488]: cluster 2024-02-10T08:34:28.093742+0000 mgr.y (mgr.14547) 476 : cluster [DBG] pgmap v677: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-10T08:34:31.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:31 smithi134 bash[18722]: cluster 2024-02-10T08:34:30.094605+0000 mgr.y (mgr.14547) 477 : cluster [DBG] pgmap v678: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 517 B/s wr, 3 op/s 2024-02-10T08:34:31.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:31 smithi134 bash[18722]: cluster 2024-02-10T08:34:31.129060+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-10T08:34:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:31 smithi067 bash[17385]: cluster 2024-02-10T08:34:30.094605+0000 mgr.y (mgr.14547) 477 : cluster [DBG] pgmap v678: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 517 B/s wr, 3 op/s 2024-02-10T08:34:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:31 smithi067 bash[17385]: cluster 2024-02-10T08:34:31.129060+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-10T08:34:31.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:31 smithi067 bash[24488]: cluster 2024-02-10T08:34:30.094605+0000 mgr.y (mgr.14547) 477 : cluster [DBG] pgmap v678: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 517 B/s wr, 3 op/s 2024-02-10T08:34:31.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:31 smithi067 bash[24488]: cluster 2024-02-10T08:34:31.129060+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-10T08:34:32.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:34:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:34:33.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:33 smithi134 bash[18722]: cluster 2024-02-10T08:34:32.096873+0000 mgr.y (mgr.14547) 478 : cluster [DBG] pgmap v680: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-10T08:34:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:33 smithi134 bash[18722]: audit 2024-02-10T08:34:32.155035+0000 mgr.y (mgr.14547) 479 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:33.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:34:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:34:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:33 smithi067 bash[17385]: cluster 2024-02-10T08:34:32.096873+0000 mgr.y (mgr.14547) 478 : cluster [DBG] pgmap v680: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-10T08:34:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:33 smithi067 bash[17385]: audit 2024-02-10T08:34:32.155035+0000 mgr.y (mgr.14547) 479 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:33 smithi067 bash[24488]: cluster 2024-02-10T08:34:32.096873+0000 mgr.y (mgr.14547) 478 : cluster [DBG] pgmap v680: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-10T08:34:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:33 smithi067 bash[24488]: audit 2024-02-10T08:34:32.155035+0000 mgr.y (mgr.14547) 479 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:35.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:35 smithi134 bash[18722]: cluster 2024-02-10T08:34:34.097562+0000 mgr.y (mgr.14547) 480 : cluster [DBG] pgmap v681: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 413 B/s wr, 3 op/s 2024-02-10T08:34:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:35 smithi134 bash[18722]: audit 2024-02-10T08:34:34.238581+0000 mon.c (mon.2) 765 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:35 smithi134 bash[18722]: audit 2024-02-10T08:34:34.239101+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:35 smithi134 bash[18722]: audit 2024-02-10T08:34:34.239899+0000 mon.c (mon.2) 766 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-63"}]: dispatch 2024-02-10T08:34:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:35 smithi134 bash[18722]: audit 2024-02-10T08:34:34.240327+0000 mon.a (mon.0) 2261 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-63"}]: dispatch 2024-02-10T08:34:35.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[24488]: cluster 2024-02-10T08:34:34.097562+0000 mgr.y (mgr.14547) 480 : cluster [DBG] pgmap v681: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 413 B/s wr, 3 op/s 2024-02-10T08:34:35.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[24488]: audit 2024-02-10T08:34:34.238581+0000 mon.c (mon.2) 765 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:35.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[24488]: audit 2024-02-10T08:34:34.239101+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:35.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[24488]: audit 2024-02-10T08:34:34.239899+0000 mon.c (mon.2) 766 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-63"}]: dispatch 2024-02-10T08:34:35.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[24488]: audit 2024-02-10T08:34:34.240327+0000 mon.a (mon.0) 2261 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-63"}]: dispatch 2024-02-10T08:34:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[17385]: cluster 2024-02-10T08:34:34.097562+0000 mgr.y (mgr.14547) 480 : cluster [DBG] pgmap v681: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 413 B/s wr, 3 op/s 2024-02-10T08:34:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[17385]: audit 2024-02-10T08:34:34.238581+0000 mon.c (mon.2) 765 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[17385]: audit 2024-02-10T08:34:34.239101+0000 mon.a (mon.0) 2260 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[17385]: audit 2024-02-10T08:34:34.239899+0000 mon.c (mon.2) 766 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-63"}]: dispatch 2024-02-10T08:34:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:35 smithi067 bash[17385]: audit 2024-02-10T08:34:34.240327+0000 mon.a (mon.0) 2261 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-63"}]: dispatch 2024-02-10T08:34:36.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:36 smithi134 bash[18722]: cluster 2024-02-10T08:34:35.236532+0000 mon.a (mon.0) 2262 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-10T08:34:36.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:36 smithi067 bash[17385]: cluster 2024-02-10T08:34:35.236532+0000 mon.a (mon.0) 2262 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-10T08:34:36.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:36 smithi067 bash[24488]: cluster 2024-02-10T08:34:35.236532+0000 mon.a (mon.0) 2262 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-10T08:34:37.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:37 smithi134 bash[18722]: cluster 2024-02-10T08:34:36.098274+0000 mgr.y (mgr.14547) 481 : cluster [DBG] pgmap v683: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:37 smithi134 bash[18722]: cluster 2024-02-10T08:34:36.262761+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-10T08:34:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:37 smithi134 bash[18722]: audit 2024-02-10T08:34:36.270428+0000 mon.c (mon.2) 767 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:37.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:37 smithi134 bash[18722]: audit 2024-02-10T08:34:36.272797+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:37.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[17385]: cluster 2024-02-10T08:34:36.098274+0000 mgr.y (mgr.14547) 481 : cluster [DBG] pgmap v683: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:37.568 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[17385]: cluster 2024-02-10T08:34:36.262761+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-10T08:34:37.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[17385]: audit 2024-02-10T08:34:36.270428+0000 mon.c (mon.2) 767 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:37.569 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[17385]: audit 2024-02-10T08:34:36.272797+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:37.569 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[24488]: cluster 2024-02-10T08:34:36.098274+0000 mgr.y (mgr.14547) 481 : cluster [DBG] pgmap v683: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:37.569 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[24488]: cluster 2024-02-10T08:34:36.262761+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-10T08:34:37.569 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[24488]: audit 2024-02-10T08:34:36.270428+0000 mon.c (mon.2) 767 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:37.569 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[24488]: audit 2024-02-10T08:34:36.272797+0000 mon.a (mon.0) 2264 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:34:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:37] "GET /metrics HTTP/1.1" 200 188573 "" "Prometheus/2.33.4" 2024-02-10T08:34:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:38 smithi067 bash[17385]: audit 2024-02-10T08:34:37.268917+0000 mon.a (mon.0) 2265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:38 smithi067 bash[17385]: cluster 2024-02-10T08:34:37.269240+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-10T08:34:38.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:38 smithi067 bash[17385]: audit 2024-02-10T08:34:37.285369+0000 mon.c (mon.2) 768 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:38 smithi067 bash[24488]: audit 2024-02-10T08:34:37.268917+0000 mon.a (mon.0) 2265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:38 smithi067 bash[24488]: cluster 2024-02-10T08:34:37.269240+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-10T08:34:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:38 smithi067 bash[24488]: audit 2024-02-10T08:34:37.285369+0000 mon.c (mon.2) 768 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:38 smithi134 bash[18722]: audit 2024-02-10T08:34:37.268917+0000 mon.a (mon.0) 2265 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:38 smithi134 bash[18722]: cluster 2024-02-10T08:34:37.269240+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-10T08:34:38.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:38 smithi134 bash[18722]: audit 2024-02-10T08:34:37.285369+0000 mon.c (mon.2) 768 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[17385]: cluster 2024-02-10T08:34:38.099106+0000 mgr.y (mgr.14547) 482 : cluster [DBG] pgmap v686: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[17385]: cluster 2024-02-10T08:34:38.286657+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-10T08:34:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[17385]: audit 2024-02-10T08:34:38.340005+0000 mon.c (mon.2) 769 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[17385]: audit 2024-02-10T08:34:38.340811+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[17385]: audit 2024-02-10T08:34:38.341866+0000 mon.c (mon.2) 770 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-65"}]: dispatch 2024-02-10T08:34:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[17385]: audit 2024-02-10T08:34:38.342438+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-65"}]: dispatch 2024-02-10T08:34:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[24488]: cluster 2024-02-10T08:34:38.099106+0000 mgr.y (mgr.14547) 482 : cluster [DBG] pgmap v686: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[24488]: cluster 2024-02-10T08:34:38.286657+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-10T08:34:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[24488]: audit 2024-02-10T08:34:38.340005+0000 mon.c (mon.2) 769 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[24488]: audit 2024-02-10T08:34:38.340811+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[24488]: audit 2024-02-10T08:34:38.341866+0000 mon.c (mon.2) 770 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-65"}]: dispatch 2024-02-10T08:34:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:39 smithi067 bash[24488]: audit 2024-02-10T08:34:38.342438+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-65"}]: dispatch 2024-02-10T08:34:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:39 smithi134 bash[18722]: cluster 2024-02-10T08:34:38.099106+0000 mgr.y (mgr.14547) 482 : cluster [DBG] pgmap v686: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:39 smithi134 bash[18722]: cluster 2024-02-10T08:34:38.286657+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-10T08:34:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:39 smithi134 bash[18722]: audit 2024-02-10T08:34:38.340005+0000 mon.c (mon.2) 769 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:39 smithi134 bash[18722]: audit 2024-02-10T08:34:38.340811+0000 mon.a (mon.0) 2268 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:39 smithi134 bash[18722]: audit 2024-02-10T08:34:38.341866+0000 mon.c (mon.2) 770 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-65"}]: dispatch 2024-02-10T08:34:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:39 smithi134 bash[18722]: audit 2024-02-10T08:34:38.342438+0000 mon.a (mon.0) 2269 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-65"}]: dispatch 2024-02-10T08:34:40.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:40 smithi067 bash[17385]: cluster 2024-02-10T08:34:39.291768+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-10T08:34:40.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:40 smithi067 bash[24488]: cluster 2024-02-10T08:34:39.291768+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-10T08:34:40.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:40 smithi134 bash[18722]: cluster 2024-02-10T08:34:39.291768+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-10T08:34:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[17385]: cluster 2024-02-10T08:34:40.099977+0000 mgr.y (mgr.14547) 483 : cluster [DBG] pgmap v689: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:34:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[17385]: cluster 2024-02-10T08:34:40.312501+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-10T08:34:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[17385]: audit 2024-02-10T08:34:40.316134+0000 mon.c (mon.2) 771 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[17385]: audit 2024-02-10T08:34:40.318029+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[24488]: cluster 2024-02-10T08:34:40.099977+0000 mgr.y (mgr.14547) 483 : cluster [DBG] pgmap v689: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:34:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[24488]: cluster 2024-02-10T08:34:40.312501+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-10T08:34:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[24488]: audit 2024-02-10T08:34:40.316134+0000 mon.c (mon.2) 771 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:41 smithi067 bash[24488]: audit 2024-02-10T08:34:40.318029+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:41 smithi134 bash[18722]: cluster 2024-02-10T08:34:40.099977+0000 mgr.y (mgr.14547) 483 : cluster [DBG] pgmap v689: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:34:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:41 smithi134 bash[18722]: cluster 2024-02-10T08:34:40.312501+0000 mon.a (mon.0) 2271 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-10T08:34:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:41 smithi134 bash[18722]: audit 2024-02-10T08:34:40.316134+0000 mon.c (mon.2) 771 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:41.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:41 smithi134 bash[18722]: audit 2024-02-10T08:34:40.318029+0000 mon.a (mon.0) 2272 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:42.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:34:43.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.320378+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:43.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: cluster 2024-02-10T08:34:41.320950+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-10T08:34:43.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.347548+0000 mon.c (mon.2) 772 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.382479+0000 mon.c (mon.2) 773 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.383073+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.384010+0000 mon.c (mon.2) 774 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-67"}]: dispatch 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.384578+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-67"}]: dispatch 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: audit 2024-02-10T08:34:41.754101+0000 mon.c (mon.2) 775 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[17385]: cluster 2024-02-10T08:34:42.323427+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-10T08:34:43.125 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.320378+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: cluster 2024-02-10T08:34:41.320950+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.347548+0000 mon.c (mon.2) 772 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.382479+0000 mon.c (mon.2) 773 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.383073+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.384010+0000 mon.c (mon.2) 774 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-67"}]: dispatch 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.384578+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-67"}]: dispatch 2024-02-10T08:34:43.126 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: audit 2024-02-10T08:34:41.754101+0000 mon.c (mon.2) 775 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:34:43.127 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:42 smithi067 bash[24488]: cluster 2024-02-10T08:34:42.323427+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-10T08:34:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.320378+0000 mon.a (mon.0) 2273 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: cluster 2024-02-10T08:34:41.320950+0000 mon.a (mon.0) 2274 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.347548+0000 mon.c (mon.2) 772 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.382479+0000 mon.c (mon.2) 773 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.383073+0000 mon.a (mon.0) 2275 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.384010+0000 mon.c (mon.2) 774 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-67"}]: dispatch 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.384578+0000 mon.a (mon.0) 2276 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-67"}]: dispatch 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: audit 2024-02-10T08:34:41.754101+0000 mon.c (mon.2) 775 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:34:43.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:42 smithi134 bash[18722]: cluster 2024-02-10T08:34:42.323427+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-10T08:34:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:34:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:34:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[24488]: cluster 2024-02-10T08:34:42.102181+0000 mgr.y (mgr.14547) 484 : cluster [DBG] pgmap v692: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T08:34:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[24488]: audit 2024-02-10T08:34:42.163041+0000 mgr.y (mgr.14547) 485 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[24488]: cluster 2024-02-10T08:34:43.326362+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-10T08:34:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[24488]: audit 2024-02-10T08:34:43.329023+0000 mon.c (mon.2) 776 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[24488]: audit 2024-02-10T08:34:43.334316+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[17385]: cluster 2024-02-10T08:34:42.102181+0000 mgr.y (mgr.14547) 484 : cluster [DBG] pgmap v692: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T08:34:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[17385]: audit 2024-02-10T08:34:42.163041+0000 mgr.y (mgr.14547) 485 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[17385]: cluster 2024-02-10T08:34:43.326362+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-10T08:34:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[17385]: audit 2024-02-10T08:34:43.329023+0000 mon.c (mon.2) 776 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:43 smithi067 bash[17385]: audit 2024-02-10T08:34:43.334316+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:43 smithi134 bash[18722]: cluster 2024-02-10T08:34:42.102181+0000 mgr.y (mgr.14547) 484 : cluster [DBG] pgmap v692: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T08:34:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:43 smithi134 bash[18722]: audit 2024-02-10T08:34:42.163041+0000 mgr.y (mgr.14547) 485 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:43 smithi134 bash[18722]: cluster 2024-02-10T08:34:43.326362+0000 mon.a (mon.0) 2278 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-10T08:34:44.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:43 smithi134 bash[18722]: audit 2024-02-10T08:34:43.329023+0000 mon.c (mon.2) 776 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:44.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:43 smithi134 bash[18722]: audit 2024-02-10T08:34:43.334316+0000 mon.a (mon.0) 2279 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:34:45.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[17385]: cluster 2024-02-10T08:34:44.103057+0000 mgr.y (mgr.14547) 486 : cluster [DBG] pgmap v695: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:34:45.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[17385]: audit 2024-02-10T08:34:44.329832+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[17385]: cluster 2024-02-10T08:34:44.330487+0000 mon.a (mon.0) 2281 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[17385]: audit 2024-02-10T08:34:44.333950+0000 mon.c (mon.2) 777 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[17385]: audit 2024-02-10T08:34:44.338904+0000 mon.c (mon.2) 778 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[17385]: audit 2024-02-10T08:34:44.341597+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[24488]: cluster 2024-02-10T08:34:44.103057+0000 mgr.y (mgr.14547) 486 : cluster [DBG] pgmap v695: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[24488]: audit 2024-02-10T08:34:44.329832+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[24488]: cluster 2024-02-10T08:34:44.330487+0000 mon.a (mon.0) 2281 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[24488]: audit 2024-02-10T08:34:44.333950+0000 mon.c (mon.2) 777 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:45.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[24488]: audit 2024-02-10T08:34:44.338904+0000 mon.c (mon.2) 778 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:45.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:45 smithi067 bash[24488]: audit 2024-02-10T08:34:44.341597+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:45.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:45 smithi134 bash[18722]: cluster 2024-02-10T08:34:44.103057+0000 mgr.y (mgr.14547) 486 : cluster [DBG] pgmap v695: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:34:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:45 smithi134 bash[18722]: audit 2024-02-10T08:34:44.329832+0000 mon.a (mon.0) 2280 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:34:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:45 smithi134 bash[18722]: cluster 2024-02-10T08:34:44.330487+0000 mon.a (mon.0) 2281 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-10T08:34:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:45 smithi134 bash[18722]: audit 2024-02-10T08:34:44.333950+0000 mon.c (mon.2) 777 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:34:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:45 smithi134 bash[18722]: audit 2024-02-10T08:34:44.338904+0000 mon.c (mon.2) 778 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:45.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:45 smithi134 bash[18722]: audit 2024-02-10T08:34:44.341597+0000 mon.a (mon.0) 2282 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:34:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:46 smithi067 bash[17385]: audit 2024-02-10T08:34:45.338603+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:34:46.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:46 smithi067 bash[17385]: cluster 2024-02-10T08:34:45.338705+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-10T08:34:46.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:46 smithi067 bash[24488]: audit 2024-02-10T08:34:45.338603+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:34:46.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:46 smithi067 bash[24488]: cluster 2024-02-10T08:34:45.338705+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-10T08:34:46.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:46 smithi134 bash[18722]: audit 2024-02-10T08:34:45.338603+0000 mon.a (mon.0) 2283 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:34:46.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:46 smithi134 bash[18722]: cluster 2024-02-10T08:34:45.338705+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-10T08:34:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:47 smithi067 bash[17385]: cluster 2024-02-10T08:34:46.104018+0000 mgr.y (mgr.14547) 487 : cluster [DBG] pgmap v698: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:47.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:47 smithi067 bash[17385]: cluster 2024-02-10T08:34:46.351533+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-10T08:34:47.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:47 smithi067 bash[24488]: cluster 2024-02-10T08:34:46.104018+0000 mgr.y (mgr.14547) 487 : cluster [DBG] pgmap v698: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:47.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:47 smithi067 bash[24488]: cluster 2024-02-10T08:34:46.351533+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-10T08:34:47.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:47 smithi134 bash[18722]: cluster 2024-02-10T08:34:46.104018+0000 mgr.y (mgr.14547) 487 : cluster [DBG] pgmap v698: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:34:47.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:47 smithi134 bash[18722]: cluster 2024-02-10T08:34:46.351533+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-10T08:34:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:34:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:47] "GET /metrics HTTP/1.1" 200 188661 "" "Prometheus/2.33.4" 2024-02-10T08:34:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:48 smithi134 bash[18722]: audit 2024-02-10T08:34:47.416048+0000 mon.a (mon.0) 2286 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:34:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:48 smithi134 bash[18722]: audit 2024-02-10T08:34:47.718732+0000 mon.c (mon.2) 779 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:34:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:48 smithi134 bash[18722]: audit 2024-02-10T08:34:47.720006+0000 mon.c (mon.2) 780 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:34:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:48 smithi134 bash[18722]: audit 2024-02-10T08:34:47.727049+0000 mon.a (mon.0) 2287 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:34:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[17385]: audit 2024-02-10T08:34:47.416048+0000 mon.a (mon.0) 2286 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:34:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[17385]: audit 2024-02-10T08:34:47.718732+0000 mon.c (mon.2) 779 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:34:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[17385]: audit 2024-02-10T08:34:47.720006+0000 mon.c (mon.2) 780 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:34:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[17385]: audit 2024-02-10T08:34:47.727049+0000 mon.a (mon.0) 2287 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:34:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[24488]: audit 2024-02-10T08:34:47.416048+0000 mon.a (mon.0) 2286 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:34:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[24488]: audit 2024-02-10T08:34:47.718732+0000 mon.c (mon.2) 779 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:34:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[24488]: audit 2024-02-10T08:34:47.720006+0000 mon.c (mon.2) 780 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:34:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:48 smithi067 bash[24488]: audit 2024-02-10T08:34:47.727049+0000 mon.a (mon.0) 2287 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:34:50.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:49 smithi134 bash[18722]: cluster 2024-02-10T08:34:48.106187+0000 mgr.y (mgr.14547) 488 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-10T08:34:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:49 smithi067 bash[17385]: cluster 2024-02-10T08:34:48.106187+0000 mgr.y (mgr.14547) 488 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-10T08:34:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:49 smithi067 bash[24488]: cluster 2024-02-10T08:34:48.106187+0000 mgr.y (mgr.14547) 488 : cluster [DBG] pgmap v700: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-02-10T08:34:52.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:51 smithi134 bash[18722]: cluster 2024-02-10T08:34:50.107014+0000 mgr.y (mgr.14547) 489 : cluster [DBG] pgmap v701: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:34:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:51 smithi067 bash[17385]: cluster 2024-02-10T08:34:50.107014+0000 mgr.y (mgr.14547) 489 : cluster [DBG] pgmap v701: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:34:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:51 smithi067 bash[24488]: cluster 2024-02-10T08:34:50.107014+0000 mgr.y (mgr.14547) 489 : cluster [DBG] pgmap v701: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-02-10T08:34:52.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:34:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:34:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:34:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:34:54.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:53 smithi134 bash[18722]: cluster 2024-02-10T08:34:52.108894+0000 mgr.y (mgr.14547) 490 : cluster [DBG] pgmap v702: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-10T08:34:54.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:53 smithi134 bash[18722]: audit 2024-02-10T08:34:52.171044+0000 mgr.y (mgr.14547) 491 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:53 smithi067 bash[17385]: cluster 2024-02-10T08:34:52.108894+0000 mgr.y (mgr.14547) 490 : cluster [DBG] pgmap v702: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-10T08:34:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:53 smithi067 bash[17385]: audit 2024-02-10T08:34:52.171044+0000 mgr.y (mgr.14547) 491 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:54.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:53 smithi067 bash[24488]: cluster 2024-02-10T08:34:52.108894+0000 mgr.y (mgr.14547) 490 : cluster [DBG] pgmap v702: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-10T08:34:54.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:53 smithi067 bash[24488]: audit 2024-02-10T08:34:52.171044+0000 mgr.y (mgr.14547) 491 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:34:56.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:55 smithi134 bash[18722]: cluster 2024-02-10T08:34:54.109722+0000 mgr.y (mgr.14547) 492 : cluster [DBG] pgmap v703: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T08:34:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:55 smithi067 bash[17385]: cluster 2024-02-10T08:34:54.109722+0000 mgr.y (mgr.14547) 492 : cluster [DBG] pgmap v703: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T08:34:56.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:55 smithi067 bash[24488]: cluster 2024-02-10T08:34:54.109722+0000 mgr.y (mgr.14547) 492 : cluster [DBG] pgmap v703: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-10T08:34:58.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:57 smithi134 bash[18722]: cluster 2024-02-10T08:34:56.110610+0000 mgr.y (mgr.14547) 493 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-10T08:34:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:34:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:34:57] "GET /metrics HTTP/1.1" 200 188661 "" "Prometheus/2.33.4" 2024-02-10T08:34:58.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:57 smithi067 bash[17385]: cluster 2024-02-10T08:34:56.110610+0000 mgr.y (mgr.14547) 493 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-10T08:34:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:57 smithi067 bash[24488]: cluster 2024-02-10T08:34:56.110610+0000 mgr.y (mgr.14547) 493 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-10T08:35:00.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:34:59 smithi134 bash[18722]: cluster 2024-02-10T08:34:58.112843+0000 mgr.y (mgr.14547) 494 : cluster [DBG] pgmap v705: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-10T08:35:00.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:34:59 smithi067 bash[17385]: cluster 2024-02-10T08:34:58.112843+0000 mgr.y (mgr.14547) 494 : cluster [DBG] pgmap v705: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-10T08:35:00.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:34:59 smithi067 bash[24488]: cluster 2024-02-10T08:34:58.112843+0000 mgr.y (mgr.14547) 494 : cluster [DBG] pgmap v705: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-10T08:35:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:01 smithi067 bash[17385]: cluster 2024-02-10T08:35:00.113522+0000 mgr.y (mgr.14547) 495 : cluster [DBG] pgmap v706: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:01.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:01 smithi067 bash[24488]: cluster 2024-02-10T08:35:00.113522+0000 mgr.y (mgr.14547) 495 : cluster [DBG] pgmap v706: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:02.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:01 smithi134 bash[18722]: cluster 2024-02-10T08:35:00.113522+0000 mgr.y (mgr.14547) 495 : cluster [DBG] pgmap v706: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:02.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:35:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:35:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:35:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:35:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:03 smithi067 bash[24488]: cluster 2024-02-10T08:35:02.115605+0000 mgr.y (mgr.14547) 496 : cluster [DBG] pgmap v707: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:03 smithi067 bash[24488]: audit 2024-02-10T08:35:02.179050+0000 mgr.y (mgr.14547) 497 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:03 smithi067 bash[17385]: cluster 2024-02-10T08:35:02.115605+0000 mgr.y (mgr.14547) 496 : cluster [DBG] pgmap v707: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:04.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:03 smithi067 bash[17385]: audit 2024-02-10T08:35:02.179050+0000 mgr.y (mgr.14547) 497 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:04.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:03 smithi134 bash[18722]: cluster 2024-02-10T08:35:02.115605+0000 mgr.y (mgr.14547) 496 : cluster [DBG] pgmap v707: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:04.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:03 smithi134 bash[18722]: audit 2024-02-10T08:35:02.179050+0000 mgr.y (mgr.14547) 497 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:05 smithi067 bash[17385]: cluster 2024-02-10T08:35:04.116363+0000 mgr.y (mgr.14547) 498 : cluster [DBG] pgmap v708: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:06.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:05 smithi067 bash[24488]: cluster 2024-02-10T08:35:04.116363+0000 mgr.y (mgr.14547) 498 : cluster [DBG] pgmap v708: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:05 smithi134 bash[18722]: cluster 2024-02-10T08:35:04.116363+0000 mgr.y (mgr.14547) 498 : cluster [DBG] pgmap v708: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:07.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:07] "GET /metrics HTTP/1.1" 200 188692 "" "Prometheus/2.33.4" 2024-02-10T08:35:07.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[17385]: cluster 2024-02-10T08:35:06.117326+0000 mgr.y (mgr.14547) 499 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:07.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[17385]: audit 2024-02-10T08:35:06.433911+0000 mon.c (mon.2) 781 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:07.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[17385]: audit 2024-02-10T08:35:06.434510+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:07.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[17385]: audit 2024-02-10T08:35:06.435603+0000 mon.c (mon.2) 782 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-69"}]: dispatch 2024-02-10T08:35:07.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[17385]: audit 2024-02-10T08:35:06.436109+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-69"}]: dispatch 2024-02-10T08:35:07.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[24488]: cluster 2024-02-10T08:35:06.117326+0000 mgr.y (mgr.14547) 499 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:07.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[24488]: audit 2024-02-10T08:35:06.433911+0000 mon.c (mon.2) 781 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:07.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[24488]: audit 2024-02-10T08:35:06.434510+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:07.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[24488]: audit 2024-02-10T08:35:06.435603+0000 mon.c (mon.2) 782 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-69"}]: dispatch 2024-02-10T08:35:07.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:07 smithi067 bash[24488]: audit 2024-02-10T08:35:06.436109+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-69"}]: dispatch 2024-02-10T08:35:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:07 smithi134 bash[18722]: cluster 2024-02-10T08:35:06.117326+0000 mgr.y (mgr.14547) 499 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:35:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:07 smithi134 bash[18722]: audit 2024-02-10T08:35:06.433911+0000 mon.c (mon.2) 781 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:07 smithi134 bash[18722]: audit 2024-02-10T08:35:06.434510+0000 mon.a (mon.0) 2288 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:07 smithi134 bash[18722]: audit 2024-02-10T08:35:06.435603+0000 mon.c (mon.2) 782 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-69"}]: dispatch 2024-02-10T08:35:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:07 smithi134 bash[18722]: audit 2024-02-10T08:35:06.436109+0000 mon.a (mon.0) 2289 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-69"}]: dispatch 2024-02-10T08:35:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[24488]: cluster 2024-02-10T08:35:07.148230+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-10T08:35:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[24488]: cluster 2024-02-10T08:35:08.150867+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-10T08:35:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[24488]: audit 2024-02-10T08:35:08.153907+0000 mon.c (mon.2) 783 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[24488]: audit 2024-02-10T08:35:08.155068+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[17385]: cluster 2024-02-10T08:35:07.148230+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-10T08:35:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[17385]: cluster 2024-02-10T08:35:08.150867+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-10T08:35:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[17385]: audit 2024-02-10T08:35:08.153907+0000 mon.c (mon.2) 783 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:08 smithi067 bash[17385]: audit 2024-02-10T08:35:08.155068+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:08 smithi134 bash[18722]: cluster 2024-02-10T08:35:07.148230+0000 mon.a (mon.0) 2290 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-10T08:35:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:08 smithi134 bash[18722]: cluster 2024-02-10T08:35:08.150867+0000 mon.a (mon.0) 2291 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-10T08:35:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:08 smithi134 bash[18722]: audit 2024-02-10T08:35:08.153907+0000 mon.c (mon.2) 783 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:09.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:08 smithi134 bash[18722]: audit 2024-02-10T08:35:08.155068+0000 mon.a (mon.0) 2292 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:09 smithi134 bash[18722]: cluster 2024-02-10T08:35:08.118036+0000 mgr.y (mgr.14547) 500 : cluster [DBG] pgmap v711: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:09 smithi134 bash[18722]: audit 2024-02-10T08:35:09.154928+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:09 smithi134 bash[18722]: cluster 2024-02-10T08:35:09.155425+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-10T08:35:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:09 smithi134 bash[18722]: audit 2024-02-10T08:35:09.163851+0000 mon.c (mon.2) 784 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[17385]: cluster 2024-02-10T08:35:08.118036+0000 mgr.y (mgr.14547) 500 : cluster [DBG] pgmap v711: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[17385]: audit 2024-02-10T08:35:09.154928+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[17385]: cluster 2024-02-10T08:35:09.155425+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-10T08:35:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[17385]: audit 2024-02-10T08:35:09.163851+0000 mon.c (mon.2) 784 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[24488]: cluster 2024-02-10T08:35:08.118036+0000 mgr.y (mgr.14547) 500 : cluster [DBG] pgmap v711: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[24488]: audit 2024-02-10T08:35:09.154928+0000 mon.a (mon.0) 2293 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[24488]: cluster 2024-02-10T08:35:09.155425+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-10T08:35:10.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:09 smithi067 bash[24488]: audit 2024-02-10T08:35:09.163851+0000 mon.c (mon.2) 784 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:11 smithi134 bash[18722]: cluster 2024-02-10T08:35:10.118901+0000 mgr.y (mgr.14547) 501 : cluster [DBG] pgmap v714: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:11 smithi134 bash[18722]: cluster 2024-02-10T08:35:10.155916+0000 mon.a (mon.0) 2295 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-10T08:35:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:11 smithi067 bash[17385]: cluster 2024-02-10T08:35:10.118901+0000 mgr.y (mgr.14547) 501 : cluster [DBG] pgmap v714: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:11 smithi067 bash[17385]: cluster 2024-02-10T08:35:10.155916+0000 mon.a (mon.0) 2295 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-10T08:35:11.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:11 smithi067 bash[24488]: cluster 2024-02-10T08:35:10.118901+0000 mgr.y (mgr.14547) 501 : cluster [DBG] pgmap v714: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:11.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:11 smithi067 bash[24488]: cluster 2024-02-10T08:35:10.155916+0000 mon.a (mon.0) 2295 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-10T08:35:12.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:12 smithi134 bash[18722]: cluster 2024-02-10T08:35:11.158272+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-10T08:35:12.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:35:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:35:12.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:12 smithi067 bash[17385]: cluster 2024-02-10T08:35:11.158272+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-10T08:35:12.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:12 smithi067 bash[24488]: cluster 2024-02-10T08:35:11.158272+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-10T08:35:13.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:13 smithi134 bash[18722]: cluster 2024-02-10T08:35:12.122959+0000 mgr.y (mgr.14547) 502 : cluster [DBG] pgmap v717: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T08:35:13.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:13 smithi134 bash[18722]: audit 2024-02-10T08:35:12.187005+0000 mgr.y (mgr.14547) 503 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:13.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:35:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:35:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:13 smithi067 bash[24488]: cluster 2024-02-10T08:35:12.122959+0000 mgr.y (mgr.14547) 502 : cluster [DBG] pgmap v717: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T08:35:13.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:13 smithi067 bash[24488]: audit 2024-02-10T08:35:12.187005+0000 mgr.y (mgr.14547) 503 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:13 smithi067 bash[17385]: cluster 2024-02-10T08:35:12.122959+0000 mgr.y (mgr.14547) 502 : cluster [DBG] pgmap v717: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-10T08:35:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:13 smithi067 bash[17385]: audit 2024-02-10T08:35:12.187005+0000 mgr.y (mgr.14547) 503 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:15 smithi134 bash[18722]: cluster 2024-02-10T08:35:14.123744+0000 mgr.y (mgr.14547) 504 : cluster [DBG] pgmap v718: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T08:35:15.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:15 smithi067 bash[24488]: cluster 2024-02-10T08:35:14.123744+0000 mgr.y (mgr.14547) 504 : cluster [DBG] pgmap v718: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T08:35:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:15 smithi067 bash[17385]: cluster 2024-02-10T08:35:14.123744+0000 mgr.y (mgr.14547) 504 : cluster [DBG] pgmap v718: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-10T08:35:17.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:17 smithi134 bash[18722]: cluster 2024-02-10T08:35:16.124501+0000 mgr.y (mgr.14547) 505 : cluster [DBG] pgmap v719: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-10T08:35:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:17 smithi067 bash[17385]: cluster 2024-02-10T08:35:16.124501+0000 mgr.y (mgr.14547) 505 : cluster [DBG] pgmap v719: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-10T08:35:17.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:17 smithi067 bash[24488]: cluster 2024-02-10T08:35:16.124501+0000 mgr.y (mgr.14547) 505 : cluster [DBG] pgmap v719: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-10T08:35:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:35:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:17] "GET /metrics HTTP/1.1" 200 188820 "" "Prometheus/2.33.4" 2024-02-10T08:35:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:19 smithi134 bash[18722]: cluster 2024-02-10T08:35:18.126701+0000 mgr.y (mgr.14547) 506 : cluster [DBG] pgmap v720: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-10T08:35:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:19 smithi067 bash[17385]: cluster 2024-02-10T08:35:18.126701+0000 mgr.y (mgr.14547) 506 : cluster [DBG] pgmap v720: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-10T08:35:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:19 smithi067 bash[24488]: cluster 2024-02-10T08:35:18.126701+0000 mgr.y (mgr.14547) 506 : cluster [DBG] pgmap v720: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-02-10T08:35:21.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:21 smithi134 bash[18722]: cluster 2024-02-10T08:35:20.127546+0000 mgr.y (mgr.14547) 507 : cluster [DBG] pgmap v721: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 718 B/s wr, 3 op/s 2024-02-10T08:35:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:21 smithi067 bash[17385]: cluster 2024-02-10T08:35:20.127546+0000 mgr.y (mgr.14547) 507 : cluster [DBG] pgmap v721: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 718 B/s wr, 3 op/s 2024-02-10T08:35:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:21 smithi067 bash[24488]: cluster 2024-02-10T08:35:20.127546+0000 mgr.y (mgr.14547) 507 : cluster [DBG] pgmap v721: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 718 B/s wr, 3 op/s 2024-02-10T08:35:22.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:35:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:35:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:22 smithi134 bash[18722]: audit 2024-02-10T08:35:21.220389+0000 mon.c (mon.2) 785 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:22 smithi134 bash[18722]: audit 2024-02-10T08:35:21.221005+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:22 smithi134 bash[18722]: audit 2024-02-10T08:35:21.222047+0000 mon.c (mon.2) 786 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-71"}]: dispatch 2024-02-10T08:35:22.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:22 smithi134 bash[18722]: audit 2024-02-10T08:35:21.222583+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-71"}]: dispatch 2024-02-10T08:35:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[17385]: audit 2024-02-10T08:35:21.220389+0000 mon.c (mon.2) 785 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[17385]: audit 2024-02-10T08:35:21.221005+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[17385]: audit 2024-02-10T08:35:21.222047+0000 mon.c (mon.2) 786 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-71"}]: dispatch 2024-02-10T08:35:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[17385]: audit 2024-02-10T08:35:21.222583+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-71"}]: dispatch 2024-02-10T08:35:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[24488]: audit 2024-02-10T08:35:21.220389+0000 mon.c (mon.2) 785 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[24488]: audit 2024-02-10T08:35:21.221005+0000 mon.a (mon.0) 2297 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[24488]: audit 2024-02-10T08:35:21.222047+0000 mon.c (mon.2) 786 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-71"}]: dispatch 2024-02-10T08:35:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:22 smithi067 bash[24488]: audit 2024-02-10T08:35:21.222583+0000 mon.a (mon.0) 2298 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-71"}]: dispatch 2024-02-10T08:35:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:23 smithi134 bash[18722]: cluster 2024-02-10T08:35:22.129762+0000 mgr.y (mgr.14547) 508 : cluster [DBG] pgmap v722: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 653 B/s wr, 3 op/s 2024-02-10T08:35:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:23 smithi134 bash[18722]: audit 2024-02-10T08:35:22.198316+0000 mgr.y (mgr.14547) 509 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:23 smithi134 bash[18722]: cluster 2024-02-10T08:35:22.208812+0000 mon.a (mon.0) 2299 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-10T08:35:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:35:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:35:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:23 smithi067 bash[17385]: cluster 2024-02-10T08:35:22.129762+0000 mgr.y (mgr.14547) 508 : cluster [DBG] pgmap v722: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 653 B/s wr, 3 op/s 2024-02-10T08:35:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:23 smithi067 bash[17385]: audit 2024-02-10T08:35:22.198316+0000 mgr.y (mgr.14547) 509 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:23 smithi067 bash[17385]: cluster 2024-02-10T08:35:22.208812+0000 mon.a (mon.0) 2299 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-10T08:35:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:23 smithi067 bash[24488]: cluster 2024-02-10T08:35:22.129762+0000 mgr.y (mgr.14547) 508 : cluster [DBG] pgmap v722: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 653 B/s wr, 3 op/s 2024-02-10T08:35:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:23 smithi067 bash[24488]: audit 2024-02-10T08:35:22.198316+0000 mgr.y (mgr.14547) 509 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:23 smithi067 bash[24488]: cluster 2024-02-10T08:35:22.208812+0000 mon.a (mon.0) 2299 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-10T08:35:24.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:24 smithi134 bash[18722]: cluster 2024-02-10T08:35:23.219899+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-10T08:35:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:24 smithi134 bash[18722]: audit 2024-02-10T08:35:23.222719+0000 mon.c (mon.2) 787 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:24 smithi134 bash[18722]: audit 2024-02-10T08:35:23.223379+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:24 smithi067 bash[17385]: cluster 2024-02-10T08:35:23.219899+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-10T08:35:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:24 smithi067 bash[17385]: audit 2024-02-10T08:35:23.222719+0000 mon.c (mon.2) 787 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:24.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:24 smithi067 bash[17385]: audit 2024-02-10T08:35:23.223379+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:24 smithi067 bash[24488]: cluster 2024-02-10T08:35:23.219899+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-10T08:35:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:24 smithi067 bash[24488]: audit 2024-02-10T08:35:23.222719+0000 mon.c (mon.2) 787 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:24 smithi067 bash[24488]: audit 2024-02-10T08:35:23.223379+0000 mon.a (mon.0) 2301 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:25 smithi134 bash[18722]: cluster 2024-02-10T08:35:24.130618+0000 mgr.y (mgr.14547) 510 : cluster [DBG] pgmap v725: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:35:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:25 smithi134 bash[18722]: audit 2024-02-10T08:35:24.236686+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:25 smithi134 bash[18722]: cluster 2024-02-10T08:35:24.237016+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-10T08:35:25.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:25 smithi134 bash[18722]: audit 2024-02-10T08:35:24.239533+0000 mon.c (mon.2) 788 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[17385]: cluster 2024-02-10T08:35:24.130618+0000 mgr.y (mgr.14547) 510 : cluster [DBG] pgmap v725: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:35:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[17385]: audit 2024-02-10T08:35:24.236686+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[17385]: cluster 2024-02-10T08:35:24.237016+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-10T08:35:25.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[17385]: audit 2024-02-10T08:35:24.239533+0000 mon.c (mon.2) 788 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[24488]: cluster 2024-02-10T08:35:24.130618+0000 mgr.y (mgr.14547) 510 : cluster [DBG] pgmap v725: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:35:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[24488]: audit 2024-02-10T08:35:24.236686+0000 mon.a (mon.0) 2302 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[24488]: cluster 2024-02-10T08:35:24.237016+0000 mon.a (mon.0) 2303 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-10T08:35:25.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:25 smithi067 bash[24488]: audit 2024-02-10T08:35:24.239533+0000 mon.c (mon.2) 788 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:26 smithi134 bash[18722]: cluster 2024-02-10T08:35:25.238758+0000 mon.a (mon.0) 2304 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-10T08:35:26.527 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:35:26 smithi134 bash[26946]: debug 2024-02-10T08:35:26.289+0000 7f8689045700 -1 snap_mapper.add_oid found existing snaps mapped on 63:cada23ff:test-rados-api-smithi067-54339-74::foo:24, removing 2024-02-10T08:35:26.527 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:35:26 smithi134 bash[36245]: debug 2024-02-10T08:35:26.289+0000 7f64fcedc700 -1 snap_mapper.add_oid found existing snaps mapped on 63:cada23ff:test-rados-api-smithi067-54339-74::foo:24, removing 2024-02-10T08:35:26.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:26 smithi067 bash[24488]: cluster 2024-02-10T08:35:25.238758+0000 mon.a (mon.0) 2304 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-10T08:35:26.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:26 smithi067 bash[17385]: cluster 2024-02-10T08:35:25.238758+0000 mon.a (mon.0) 2304 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-10T08:35:26.621 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:35:26 smithi067 bash[37094]: debug 2024-02-10T08:35:26.287+0000 7f033b94a700 -1 snap_mapper.add_oid found existing snaps mapped on 63:cada23ff:test-rados-api-smithi067-54339-74::foo:24, removing 2024-02-10T08:35:27.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:27 smithi134 bash[18722]: cluster 2024-02-10T08:35:26.131432+0000 mgr.y (mgr.14547) 511 : cluster [DBG] pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:27 smithi134 bash[18722]: cluster 2024-02-10T08:35:26.250662+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-10T08:35:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:27 smithi134 bash[18722]: audit 2024-02-10T08:35:26.323394+0000 mon.c (mon.2) 789 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:27 smithi134 bash[18722]: audit 2024-02-10T08:35:26.324041+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:27 smithi134 bash[18722]: audit 2024-02-10T08:35:26.325108+0000 mon.c (mon.2) 790 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-73"}]: dispatch 2024-02-10T08:35:27.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:27 smithi134 bash[18722]: audit 2024-02-10T08:35:26.325647+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-73"}]: dispatch 2024-02-10T08:35:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[24488]: cluster 2024-02-10T08:35:26.131432+0000 mgr.y (mgr.14547) 511 : cluster [DBG] pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[24488]: cluster 2024-02-10T08:35:26.250662+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-10T08:35:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[24488]: audit 2024-02-10T08:35:26.323394+0000 mon.c (mon.2) 789 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[24488]: audit 2024-02-10T08:35:26.324041+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[24488]: audit 2024-02-10T08:35:26.325108+0000 mon.c (mon.2) 790 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-73"}]: dispatch 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[24488]: audit 2024-02-10T08:35:26.325647+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-73"}]: dispatch 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17385]: cluster 2024-02-10T08:35:26.131432+0000 mgr.y (mgr.14547) 511 : cluster [DBG] pgmap v728: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17385]: cluster 2024-02-10T08:35:26.250662+0000 mon.a (mon.0) 2305 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17385]: audit 2024-02-10T08:35:26.323394+0000 mon.c (mon.2) 789 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17385]: audit 2024-02-10T08:35:26.324041+0000 mon.a (mon.0) 2306 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17385]: audit 2024-02-10T08:35:26.325108+0000 mon.c (mon.2) 790 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-73"}]: dispatch 2024-02-10T08:35:27.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17385]: audit 2024-02-10T08:35:26.325647+0000 mon.a (mon.0) 2307 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-73"}]: dispatch 2024-02-10T08:35:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:35:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:27] "GET /metrics HTTP/1.1" 200 188820 "" "Prometheus/2.33.4" 2024-02-10T08:35:28.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:28 smithi067 bash[24488]: cluster 2024-02-10T08:35:27.270965+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-10T08:35:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:28 smithi067 bash[17385]: cluster 2024-02-10T08:35:27.270965+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-10T08:35:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:28 smithi134 bash[18722]: cluster 2024-02-10T08:35:27.270965+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-10T08:35:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[17385]: cluster 2024-02-10T08:35:28.132469+0000 mgr.y (mgr.14547) 512 : cluster [DBG] pgmap v731: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T08:35:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[17385]: cluster 2024-02-10T08:35:28.286871+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-10T08:35:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[17385]: audit 2024-02-10T08:35:28.288968+0000 mon.c (mon.2) 791 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:29.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[17385]: audit 2024-02-10T08:35:28.289713+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[24488]: cluster 2024-02-10T08:35:28.132469+0000 mgr.y (mgr.14547) 512 : cluster [DBG] pgmap v731: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T08:35:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[24488]: cluster 2024-02-10T08:35:28.286871+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-10T08:35:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[24488]: audit 2024-02-10T08:35:28.288968+0000 mon.c (mon.2) 791 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:29 smithi067 bash[24488]: audit 2024-02-10T08:35:28.289713+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:29 smithi134 bash[18722]: cluster 2024-02-10T08:35:28.132469+0000 mgr.y (mgr.14547) 512 : cluster [DBG] pgmap v731: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T08:35:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:29 smithi134 bash[18722]: cluster 2024-02-10T08:35:28.286871+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-10T08:35:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:29 smithi134 bash[18722]: audit 2024-02-10T08:35:28.288968+0000 mon.c (mon.2) 791 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:29 smithi134 bash[18722]: audit 2024-02-10T08:35:28.289713+0000 mon.a (mon.0) 2310 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:30.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:30 smithi067 bash[24488]: audit 2024-02-10T08:35:29.293225+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:30.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:30 smithi067 bash[24488]: cluster 2024-02-10T08:35:29.293882+0000 mon.a (mon.0) 2312 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-10T08:35:30.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:30 smithi067 bash[24488]: audit 2024-02-10T08:35:29.305245+0000 mon.c (mon.2) 792 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:30 smithi067 bash[17385]: audit 2024-02-10T08:35:29.293225+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:30 smithi067 bash[17385]: cluster 2024-02-10T08:35:29.293882+0000 mon.a (mon.0) 2312 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-10T08:35:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:30 smithi067 bash[17385]: audit 2024-02-10T08:35:29.305245+0000 mon.c (mon.2) 792 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:30.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:30 smithi134 bash[18722]: audit 2024-02-10T08:35:29.293225+0000 mon.a (mon.0) 2311 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:30 smithi134 bash[18722]: cluster 2024-02-10T08:35:29.293882+0000 mon.a (mon.0) 2312 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-10T08:35:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:30 smithi134 bash[18722]: audit 2024-02-10T08:35:29.305245+0000 mon.c (mon.2) 792 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:31.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:31 smithi067 bash[24488]: cluster 2024-02-10T08:35:30.133344+0000 mgr.y (mgr.14547) 513 : cluster [DBG] pgmap v734: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T08:35:31.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:31 smithi067 bash[24488]: cluster 2024-02-10T08:35:30.307494+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-10T08:35:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:31 smithi067 bash[17385]: cluster 2024-02-10T08:35:30.133344+0000 mgr.y (mgr.14547) 513 : cluster [DBG] pgmap v734: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T08:35:31.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:31 smithi067 bash[17385]: cluster 2024-02-10T08:35:30.307494+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-10T08:35:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:31 smithi134 bash[18722]: cluster 2024-02-10T08:35:30.133344+0000 mgr.y (mgr.14547) 513 : cluster [DBG] pgmap v734: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-10T08:35:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:31 smithi134 bash[18722]: cluster 2024-02-10T08:35:30.307494+0000 mon.a (mon.0) 2313 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-10T08:35:32.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:32 smithi134 bash[18722]: cluster 2024-02-10T08:35:31.329806+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-10T08:35:32.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:32 smithi134 bash[18722]: audit 2024-02-10T08:35:31.354419+0000 mon.c (mon.2) 793 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.19"}]: dispatch 2024-02-10T08:35:32.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:32 smithi134 bash[18722]: audit 2024-02-10T08:35:31.354684+0000 mgr.y (mgr.14547) 514 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.19"}]: dispatch 2024-02-10T08:35:32.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:35:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:35:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:32 smithi067 bash[17385]: cluster 2024-02-10T08:35:31.329806+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-10T08:35:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:32 smithi067 bash[17385]: audit 2024-02-10T08:35:31.354419+0000 mon.c (mon.2) 793 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.19"}]: dispatch 2024-02-10T08:35:32.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:32 smithi067 bash[17385]: audit 2024-02-10T08:35:31.354684+0000 mgr.y (mgr.14547) 514 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.19"}]: dispatch 2024-02-10T08:35:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:32 smithi067 bash[24488]: cluster 2024-02-10T08:35:31.329806+0000 mon.a (mon.0) 2314 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-10T08:35:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:32 smithi067 bash[24488]: audit 2024-02-10T08:35:31.354419+0000 mon.c (mon.2) 793 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.19"}]: dispatch 2024-02-10T08:35:32.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:32 smithi067 bash[24488]: audit 2024-02-10T08:35:31.354684+0000 mgr.y (mgr.14547) 514 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.19"}]: dispatch 2024-02-10T08:35:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:35:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:35:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:33 smithi134 bash[18722]: cluster 2024-02-10T08:35:32.048468+0000 osd.5 (osd.5) 9 : cluster [DBG] 219.19 deep-scrub starts 2024-02-10T08:35:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:33 smithi134 bash[18722]: cluster 2024-02-10T08:35:32.053152+0000 osd.5 (osd.5) 10 : cluster [DBG] 219.19 deep-scrub ok 2024-02-10T08:35:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:33 smithi134 bash[18722]: cluster 2024-02-10T08:35:32.135291+0000 mgr.y (mgr.14547) 515 : cluster [DBG] pgmap v737: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T08:35:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:33 smithi134 bash[18722]: audit 2024-02-10T08:35:32.207120+0000 mgr.y (mgr.14547) 516 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[17385]: cluster 2024-02-10T08:35:32.048468+0000 osd.5 (osd.5) 9 : cluster [DBG] 219.19 deep-scrub starts 2024-02-10T08:35:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[17385]: cluster 2024-02-10T08:35:32.053152+0000 osd.5 (osd.5) 10 : cluster [DBG] 219.19 deep-scrub ok 2024-02-10T08:35:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[17385]: cluster 2024-02-10T08:35:32.135291+0000 mgr.y (mgr.14547) 515 : cluster [DBG] pgmap v737: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T08:35:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[17385]: audit 2024-02-10T08:35:32.207120+0000 mgr.y (mgr.14547) 516 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[24488]: cluster 2024-02-10T08:35:32.048468+0000 osd.5 (osd.5) 9 : cluster [DBG] 219.19 deep-scrub starts 2024-02-10T08:35:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[24488]: cluster 2024-02-10T08:35:32.053152+0000 osd.5 (osd.5) 10 : cluster [DBG] 219.19 deep-scrub ok 2024-02-10T08:35:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[24488]: cluster 2024-02-10T08:35:32.135291+0000 mgr.y (mgr.14547) 515 : cluster [DBG] pgmap v737: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-10T08:35:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:33 smithi067 bash[24488]: audit 2024-02-10T08:35:32.207120+0000 mgr.y (mgr.14547) 516 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:35 smithi067 bash[17385]: cluster 2024-02-10T08:35:34.136115+0000 mgr.y (mgr.14547) 517 : cluster [DBG] pgmap v738: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 874 B/s rd, 874 B/s wr, 2 op/s 2024-02-10T08:35:35.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:35 smithi067 bash[24488]: cluster 2024-02-10T08:35:34.136115+0000 mgr.y (mgr.14547) 517 : cluster [DBG] pgmap v738: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 874 B/s rd, 874 B/s wr, 2 op/s 2024-02-10T08:35:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:35 smithi134 bash[18722]: cluster 2024-02-10T08:35:34.136115+0000 mgr.y (mgr.14547) 517 : cluster [DBG] pgmap v738: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 874 B/s rd, 874 B/s wr, 2 op/s 2024-02-10T08:35:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:37 smithi067 bash[17385]: cluster 2024-02-10T08:35:36.136937+0000 mgr.y (mgr.14547) 518 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 747 B/s rd, 747 B/s wr, 2 op/s 2024-02-10T08:35:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:37 smithi067 bash[24488]: cluster 2024-02-10T08:35:36.136937+0000 mgr.y (mgr.14547) 518 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 747 B/s rd, 747 B/s wr, 2 op/s 2024-02-10T08:35:37.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:37 smithi134 bash[18722]: cluster 2024-02-10T08:35:36.136937+0000 mgr.y (mgr.14547) 518 : cluster [DBG] pgmap v739: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 747 B/s rd, 747 B/s wr, 2 op/s 2024-02-10T08:35:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:35:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:37] "GET /metrics HTTP/1.1" 200 188914 "" "Prometheus/2.33.4" 2024-02-10T08:35:39.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:39 smithi067 bash[24488]: cluster 2024-02-10T08:35:38.139214+0000 mgr.y (mgr.14547) 519 : cluster [DBG] pgmap v740: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-10T08:35:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:39 smithi067 bash[17385]: cluster 2024-02-10T08:35:38.139214+0000 mgr.y (mgr.14547) 519 : cluster [DBG] pgmap v740: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-10T08:35:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:39 smithi134 bash[18722]: cluster 2024-02-10T08:35:38.139214+0000 mgr.y (mgr.14547) 519 : cluster [DBG] pgmap v740: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-02-10T08:35:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:41 smithi067 bash[24488]: cluster 2024-02-10T08:35:40.139854+0000 mgr.y (mgr.14547) 520 : cluster [DBG] pgmap v741: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 520 B/s wr, 2 op/s 2024-02-10T08:35:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:41 smithi067 bash[17385]: cluster 2024-02-10T08:35:40.139854+0000 mgr.y (mgr.14547) 520 : cluster [DBG] pgmap v741: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 520 B/s wr, 2 op/s 2024-02-10T08:35:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:41 smithi134 bash[18722]: cluster 2024-02-10T08:35:40.139854+0000 mgr.y (mgr.14547) 520 : cluster [DBG] pgmap v741: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 520 B/s wr, 2 op/s 2024-02-10T08:35:42.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:35:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:35:43.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:43 smithi134 bash[18722]: cluster 2024-02-10T08:35:42.142077+0000 mgr.y (mgr.14547) 521 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 473 B/s wr, 2 op/s 2024-02-10T08:35:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:43 smithi134 bash[18722]: audit 2024-02-10T08:35:42.215130+0000 mgr.y (mgr.14547) 522 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:43.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:35:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:35:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:43 smithi067 bash[17385]: cluster 2024-02-10T08:35:42.142077+0000 mgr.y (mgr.14547) 521 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 473 B/s wr, 2 op/s 2024-02-10T08:35:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:43 smithi067 bash[17385]: audit 2024-02-10T08:35:42.215130+0000 mgr.y (mgr.14547) 522 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:43 smithi067 bash[24488]: cluster 2024-02-10T08:35:42.142077+0000 mgr.y (mgr.14547) 521 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 473 B/s wr, 2 op/s 2024-02-10T08:35:43.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:43 smithi067 bash[24488]: audit 2024-02-10T08:35:42.215130+0000 mgr.y (mgr.14547) 522 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:45.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:45 smithi134 bash[18722]: cluster 2024-02-10T08:35:44.142924+0000 mgr.y (mgr.14547) 523 : cluster [DBG] pgmap v743: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:45 smithi067 bash[17385]: cluster 2024-02-10T08:35:44.142924+0000 mgr.y (mgr.14547) 523 : cluster [DBG] pgmap v743: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:45.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:45 smithi067 bash[24488]: cluster 2024-02-10T08:35:44.142924+0000 mgr.y (mgr.14547) 523 : cluster [DBG] pgmap v743: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:47.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:47 smithi134 bash[18722]: cluster 2024-02-10T08:35:46.143688+0000 mgr.y (mgr.14547) 524 : cluster [DBG] pgmap v744: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:47.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:47 smithi067 bash[17385]: cluster 2024-02-10T08:35:46.143688+0000 mgr.y (mgr.14547) 524 : cluster [DBG] pgmap v744: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:47.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:47 smithi067 bash[24488]: cluster 2024-02-10T08:35:46.143688+0000 mgr.y (mgr.14547) 524 : cluster [DBG] pgmap v744: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:35:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:47] "GET /metrics HTTP/1.1" 200 188950 "" "Prometheus/2.33.4" 2024-02-10T08:35:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:48 smithi134 bash[18722]: audit 2024-02-10T08:35:47.736615+0000 mon.c (mon.2) 794 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:35:48.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:48 smithi134 bash[18722]: audit 2024-02-10T08:35:48.018754+0000 mon.a (mon.0) 2315 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:48 smithi067 bash[17385]: audit 2024-02-10T08:35:47.736615+0000 mon.c (mon.2) 794 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:35:48.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:48 smithi067 bash[17385]: audit 2024-02-10T08:35:48.018754+0000 mon.a (mon.0) 2315 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:48 smithi067 bash[24488]: audit 2024-02-10T08:35:47.736615+0000 mon.c (mon.2) 794 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:35:48.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:48 smithi067 bash[24488]: audit 2024-02-10T08:35:48.018754+0000 mon.a (mon.0) 2315 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:49 smithi134 bash[18722]: cluster 2024-02-10T08:35:48.145730+0000 mgr.y (mgr.14547) 525 : cluster [DBG] pgmap v745: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:35:49.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:49 smithi067 bash[24488]: cluster 2024-02-10T08:35:48.145730+0000 mgr.y (mgr.14547) 525 : cluster [DBG] pgmap v745: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:35:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:49 smithi067 bash[17385]: cluster 2024-02-10T08:35:48.145730+0000 mgr.y (mgr.14547) 525 : cluster [DBG] pgmap v745: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:35:51.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:35:51 smithi134 bash[26946]: debug 2024-02-10T08:35:51.377+0000 7f8691055700 -1 snap_mapper.add_oid found existing snaps mapped on 219:98020916:test-rados-api-smithi067-54339-76::foo:2, removing 2024-02-10T08:35:51.777 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:35:51 smithi134 bash[36245]: debug 2024-02-10T08:35:51.381+0000 7f64f9ed6700 -1 snap_mapper.add_oid found existing snaps mapped on 219:98020916:test-rados-api-smithi067-54339-76::foo:2, removing 2024-02-10T08:35:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:51 smithi134 bash[18722]: cluster 2024-02-10T08:35:50.146593+0000 mgr.y (mgr.14547) 526 : cluster [DBG] pgmap v746: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:51 smithi134 bash[18722]: audit 2024-02-10T08:35:51.384713+0000 mon.c (mon.2) 795 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:51 smithi134 bash[18722]: audit 2024-02-10T08:35:51.385307+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:51.870 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[37094]: debug 2024-02-10T08:35:51.378+0000 7f033c94c700 -1 snap_mapper.add_oid found existing snaps mapped on 219:98020916:test-rados-api-smithi067-54339-76::foo:2, removing 2024-02-10T08:35:51.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[17385]: cluster 2024-02-10T08:35:50.146593+0000 mgr.y (mgr.14547) 526 : cluster [DBG] pgmap v746: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[17385]: audit 2024-02-10T08:35:51.384713+0000 mon.c (mon.2) 795 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:51.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[17385]: audit 2024-02-10T08:35:51.385307+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:51.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[24488]: cluster 2024-02-10T08:35:50.146593+0000 mgr.y (mgr.14547) 526 : cluster [DBG] pgmap v746: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:35:51.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[24488]: audit 2024-02-10T08:35:51.384713+0000 mon.c (mon.2) 795 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:51.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:51 smithi067 bash[24488]: audit 2024-02-10T08:35:51.385307+0000 mon.a (mon.0) 2316 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:52.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:52 smithi134 bash[18722]: audit 2024-02-10T08:35:51.386399+0000 mon.c (mon.2) 796 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-75"}]: dispatch 2024-02-10T08:35:52.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:52 smithi134 bash[18722]: audit 2024-02-10T08:35:51.386902+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-75"}]: dispatch 2024-02-10T08:35:52.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:35:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:35:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:52 smithi067 bash[17385]: audit 2024-02-10T08:35:51.386399+0000 mon.c (mon.2) 796 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-75"}]: dispatch 2024-02-10T08:35:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:52 smithi067 bash[17385]: audit 2024-02-10T08:35:51.386902+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-75"}]: dispatch 2024-02-10T08:35:52.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:52 smithi067 bash[24488]: audit 2024-02-10T08:35:51.386399+0000 mon.c (mon.2) 796 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-75"}]: dispatch 2024-02-10T08:35:52.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:52 smithi067 bash[24488]: audit 2024-02-10T08:35:51.386902+0000 mon.a (mon.0) 2317 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-75"}]: dispatch 2024-02-10T08:35:53.404 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:35:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:35:53.405 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:53 smithi134 bash[18722]: cluster 2024-02-10T08:35:52.148828+0000 mgr.y (mgr.14547) 527 : cluster [DBG] pgmap v747: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T08:35:53.405 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:53 smithi134 bash[18722]: audit 2024-02-10T08:35:52.218436+0000 mgr.y (mgr.14547) 528 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:53.405 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:53 smithi134 bash[18722]: cluster 2024-02-10T08:35:52.406748+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-10T08:35:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:53 smithi067 bash[17385]: cluster 2024-02-10T08:35:52.148828+0000 mgr.y (mgr.14547) 527 : cluster [DBG] pgmap v747: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T08:35:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:53 smithi067 bash[17385]: audit 2024-02-10T08:35:52.218436+0000 mgr.y (mgr.14547) 528 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:53 smithi067 bash[17385]: cluster 2024-02-10T08:35:52.406748+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-10T08:35:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:53 smithi067 bash[24488]: cluster 2024-02-10T08:35:52.148828+0000 mgr.y (mgr.14547) 527 : cluster [DBG] pgmap v747: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 85 B/s wr, 1 op/s 2024-02-10T08:35:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:53 smithi067 bash[24488]: audit 2024-02-10T08:35:52.218436+0000 mgr.y (mgr.14547) 528 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:35:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:53 smithi067 bash[24488]: cluster 2024-02-10T08:35:52.406748+0000 mon.a (mon.0) 2318 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: cluster 2024-02-10T08:35:53.415131+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: audit 2024-02-10T08:35:53.421717+0000 mon.c (mon.2) 797 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: audit 2024-02-10T08:35:53.422405+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: audit 2024-02-10T08:35:53.729065+0000 mon.a (mon.0) 2321 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: audit 2024-02-10T08:35:54.019940+0000 mon.c (mon.2) 798 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: audit 2024-02-10T08:35:54.021779+0000 mon.c (mon.2) 799 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:35:54.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:54 smithi134 bash[18722]: audit 2024-02-10T08:35:54.031892+0000 mon.a (mon.0) 2322 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:54.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: cluster 2024-02-10T08:35:53.415131+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-10T08:35:54.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: audit 2024-02-10T08:35:53.421717+0000 mon.c (mon.2) 797 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:54.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: audit 2024-02-10T08:35:53.422405+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: audit 2024-02-10T08:35:53.729065+0000 mon.a (mon.0) 2321 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: audit 2024-02-10T08:35:54.019940+0000 mon.c (mon.2) 798 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: audit 2024-02-10T08:35:54.021779+0000 mon.c (mon.2) 799 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[24488]: audit 2024-02-10T08:35:54.031892+0000 mon.a (mon.0) 2322 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: cluster 2024-02-10T08:35:53.415131+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: audit 2024-02-10T08:35:53.421717+0000 mon.c (mon.2) 797 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: audit 2024-02-10T08:35:53.422405+0000 mon.a (mon.0) 2320 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: audit 2024-02-10T08:35:53.729065+0000 mon.a (mon.0) 2321 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: audit 2024-02-10T08:35:54.019940+0000 mon.c (mon.2) 798 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: audit 2024-02-10T08:35:54.021779+0000 mon.c (mon.2) 799 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:35:54.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:54 smithi067 bash[17385]: audit 2024-02-10T08:35:54.031892+0000 mon.a (mon.0) 2322 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:35:55.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:55 smithi134 bash[18722]: cluster 2024-02-10T08:35:54.149668+0000 mgr.y (mgr.14547) 529 : cluster [DBG] pgmap v750: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-10T08:35:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:55 smithi134 bash[18722]: audit 2024-02-10T08:35:54.420637+0000 mon.a (mon.0) 2323 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:55 smithi134 bash[18722]: cluster 2024-02-10T08:35:54.421084+0000 mon.a (mon.0) 2324 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-10T08:35:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:55 smithi134 bash[18722]: audit 2024-02-10T08:35:54.438096+0000 mon.c (mon.2) 800 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:55 smithi134 bash[18722]: audit 2024-02-10T08:35:54.442678+0000 mon.c (mon.2) 801 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:35:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:55 smithi134 bash[18722]: audit 2024-02-10T08:35:54.445011+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:35:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[17385]: cluster 2024-02-10T08:35:54.149668+0000 mgr.y (mgr.14547) 529 : cluster [DBG] pgmap v750: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[17385]: audit 2024-02-10T08:35:54.420637+0000 mon.a (mon.0) 2323 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[17385]: cluster 2024-02-10T08:35:54.421084+0000 mon.a (mon.0) 2324 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[17385]: audit 2024-02-10T08:35:54.438096+0000 mon.c (mon.2) 800 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[17385]: audit 2024-02-10T08:35:54.442678+0000 mon.c (mon.2) 801 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[17385]: audit 2024-02-10T08:35:54.445011+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[24488]: cluster 2024-02-10T08:35:54.149668+0000 mgr.y (mgr.14547) 529 : cluster [DBG] pgmap v750: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[24488]: audit 2024-02-10T08:35:54.420637+0000 mon.a (mon.0) 2323 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[24488]: cluster 2024-02-10T08:35:54.421084+0000 mon.a (mon.0) 2324 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[24488]: audit 2024-02-10T08:35:54.438096+0000 mon.c (mon.2) 800 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[24488]: audit 2024-02-10T08:35:54.442678+0000 mon.c (mon.2) 801 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:35:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:55 smithi067 bash[24488]: audit 2024-02-10T08:35:54.445011+0000 mon.a (mon.0) 2325 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:35:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:56 smithi134 bash[18722]: audit 2024-02-10T08:35:55.442582+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:35:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:56 smithi134 bash[18722]: cluster 2024-02-10T08:35:55.442693+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-10T08:35:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:56 smithi134 bash[18722]: audit 2024-02-10T08:35:55.444641+0000 mon.c (mon.2) 802 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:56 smithi134 bash[18722]: audit 2024-02-10T08:35:55.445626+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[17385]: audit 2024-02-10T08:35:55.442582+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:35:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[17385]: cluster 2024-02-10T08:35:55.442693+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-10T08:35:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[17385]: audit 2024-02-10T08:35:55.444641+0000 mon.c (mon.2) 802 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:56.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[17385]: audit 2024-02-10T08:35:55.445626+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[24488]: audit 2024-02-10T08:35:55.442582+0000 mon.a (mon.0) 2326 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:35:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[24488]: cluster 2024-02-10T08:35:55.442693+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-10T08:35:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[24488]: audit 2024-02-10T08:35:55.444641+0000 mon.c (mon.2) 802 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:56 smithi067 bash[24488]: audit 2024-02-10T08:35:55.445626+0000 mon.a (mon.0) 2328 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: cluster 2024-02-10T08:35:56.150486+0000 mgr.y (mgr.14547) 530 : cluster [DBG] pgmap v753: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: audit 2024-02-10T08:35:56.447078+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: cluster 2024-02-10T08:35:56.447905+0000 mon.a (mon.0) 2330 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: audit 2024-02-10T08:35:56.449339+0000 mon.c (mon.2) 803 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: audit 2024-02-10T08:35:56.450072+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: audit 2024-02-10T08:35:57.450021+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:35:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:57 smithi134 bash[18722]: cluster 2024-02-10T08:35:57.450167+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: cluster 2024-02-10T08:35:56.150486+0000 mgr.y (mgr.14547) 530 : cluster [DBG] pgmap v753: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: audit 2024-02-10T08:35:56.447078+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: cluster 2024-02-10T08:35:56.447905+0000 mon.a (mon.0) 2330 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: audit 2024-02-10T08:35:56.449339+0000 mon.c (mon.2) 803 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: audit 2024-02-10T08:35:56.450072+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: audit 2024-02-10T08:35:57.450021+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:35:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17385]: cluster 2024-02-10T08:35:57.450167+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: cluster 2024-02-10T08:35:56.150486+0000 mgr.y (mgr.14547) 530 : cluster [DBG] pgmap v753: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: audit 2024-02-10T08:35:56.447078+0000 mon.a (mon.0) 2329 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: cluster 2024-02-10T08:35:56.447905+0000 mon.a (mon.0) 2330 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: audit 2024-02-10T08:35:56.449339+0000 mon.c (mon.2) 803 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: audit 2024-02-10T08:35:56.450072+0000 mon.a (mon.0) 2331 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: audit 2024-02-10T08:35:57.450021+0000 mon.a (mon.0) 2332 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:35:57.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[24488]: cluster 2024-02-10T08:35:57.450167+0000 mon.a (mon.0) 2333 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-10T08:35:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:35:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:35:57] "GET /metrics HTTP/1.1" 200 188950 "" "Prometheus/2.33.4" 2024-02-10T08:35:58.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:58 smithi134 bash[18722]: audit 2024-02-10T08:35:57.451600+0000 mon.c (mon.2) 804 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:35:58.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:58 smithi134 bash[18722]: audit 2024-02-10T08:35:57.454416+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:35:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:58 smithi067 bash[17385]: audit 2024-02-10T08:35:57.451600+0000 mon.c (mon.2) 804 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:35:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:58 smithi067 bash[17385]: audit 2024-02-10T08:35:57.454416+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:35:58.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:58 smithi067 bash[24488]: audit 2024-02-10T08:35:57.451600+0000 mon.c (mon.2) 804 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:35:58.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:58 smithi067 bash[24488]: audit 2024-02-10T08:35:57.454416+0000 mon.a (mon.0) 2334 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:35:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:59 smithi134 bash[18722]: cluster 2024-02-10T08:35:58.151415+0000 mgr.y (mgr.14547) 531 : cluster [DBG] pgmap v756: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:35:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:59 smithi134 bash[18722]: audit 2024-02-10T08:35:58.453462+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:35:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:59 smithi134 bash[18722]: cluster 2024-02-10T08:35:58.453623+0000 mon.a (mon.0) 2336 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-10T08:35:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:59 smithi134 bash[18722]: audit 2024-02-10T08:35:58.484345+0000 mon.c (mon.2) 805 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T08:35:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:35:59 smithi134 bash[18722]: audit 2024-02-10T08:35:58.485020+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T08:35:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[17385]: cluster 2024-02-10T08:35:58.151415+0000 mgr.y (mgr.14547) 531 : cluster [DBG] pgmap v756: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:35:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[17385]: audit 2024-02-10T08:35:58.453462+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:35:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[17385]: cluster 2024-02-10T08:35:58.453623+0000 mon.a (mon.0) 2336 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-10T08:35:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[17385]: audit 2024-02-10T08:35:58.484345+0000 mon.c (mon.2) 805 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T08:35:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[17385]: audit 2024-02-10T08:35:58.485020+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T08:35:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[24488]: cluster 2024-02-10T08:35:58.151415+0000 mgr.y (mgr.14547) 531 : cluster [DBG] pgmap v756: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:35:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[24488]: audit 2024-02-10T08:35:58.453462+0000 mon.a (mon.0) 2335 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:35:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[24488]: cluster 2024-02-10T08:35:58.453623+0000 mon.a (mon.0) 2336 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-10T08:35:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[24488]: audit 2024-02-10T08:35:58.484345+0000 mon.c (mon.2) 805 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T08:35:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:35:59 smithi067 bash[24488]: audit 2024-02-10T08:35:58.485020+0000 mon.a (mon.0) 2337 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-10T08:36:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:00 smithi134 bash[18722]: audit 2024-02-10T08:35:59.472874+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T08:36:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:00 smithi134 bash[18722]: cluster 2024-02-10T08:35:59.473072+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-10T08:36:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:00 smithi134 bash[18722]: audit 2024-02-10T08:35:59.492771+0000 mon.c (mon.2) 806 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T08:36:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:00 smithi134 bash[18722]: audit 2024-02-10T08:35:59.493962+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T08:36:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[17385]: audit 2024-02-10T08:35:59.472874+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T08:36:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[17385]: cluster 2024-02-10T08:35:59.473072+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-10T08:36:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[17385]: audit 2024-02-10T08:35:59.492771+0000 mon.c (mon.2) 806 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T08:36:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[17385]: audit 2024-02-10T08:35:59.493962+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T08:36:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[24488]: audit 2024-02-10T08:35:59.472874+0000 mon.a (mon.0) 2338 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-10T08:36:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[24488]: cluster 2024-02-10T08:35:59.473072+0000 mon.a (mon.0) 2339 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-10T08:36:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[24488]: audit 2024-02-10T08:35:59.492771+0000 mon.c (mon.2) 806 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T08:36:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:00 smithi067 bash[24488]: audit 2024-02-10T08:35:59.493962+0000 mon.a (mon.0) 2340 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-10T08:36:01.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:01 smithi134 bash[18722]: cluster 2024-02-10T08:36:00.152252+0000 mgr.y (mgr.14547) 532 : cluster [DBG] pgmap v759: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:01 smithi134 bash[18722]: audit 2024-02-10T08:36:00.485317+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T08:36:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:01 smithi134 bash[18722]: cluster 2024-02-10T08:36:00.485434+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-10T08:36:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:01 smithi134 bash[18722]: audit 2024-02-10T08:36:00.498902+0000 mon.c (mon.2) 807 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:01 smithi134 bash[18722]: audit 2024-02-10T08:36:00.499870+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[17385]: cluster 2024-02-10T08:36:00.152252+0000 mgr.y (mgr.14547) 532 : cluster [DBG] pgmap v759: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[17385]: audit 2024-02-10T08:36:00.485317+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T08:36:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[17385]: cluster 2024-02-10T08:36:00.485434+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-10T08:36:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[17385]: audit 2024-02-10T08:36:00.498902+0000 mon.c (mon.2) 807 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[17385]: audit 2024-02-10T08:36:00.499870+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[24488]: cluster 2024-02-10T08:36:00.152252+0000 mgr.y (mgr.14547) 532 : cluster [DBG] pgmap v759: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[24488]: audit 2024-02-10T08:36:00.485317+0000 mon.a (mon.0) 2341 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-10T08:36:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[24488]: cluster 2024-02-10T08:36:00.485434+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-10T08:36:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[24488]: audit 2024-02-10T08:36:00.498902+0000 mon.c (mon.2) 807 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:01 smithi067 bash[24488]: audit 2024-02-10T08:36:00.499870+0000 mon.a (mon.0) 2343 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:02.499 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:36:02.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[18722]: audit 2024-02-10T08:36:01.500231+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:02.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[18722]: cluster 2024-02-10T08:36:01.500350+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-10T08:36:02.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[18722]: audit 2024-02-10T08:36:01.546520+0000 mon.c (mon.2) 808 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:02.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[18722]: audit 2024-02-10T08:36:01.547296+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:02.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[18722]: audit 2024-02-10T08:36:01.548312+0000 mon.c (mon.2) 809 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-77"}]: dispatch 2024-02-10T08:36:02.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:02 smithi134 bash[18722]: audit 2024-02-10T08:36:01.548912+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-77"}]: dispatch 2024-02-10T08:36:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[24488]: audit 2024-02-10T08:36:01.500231+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[24488]: cluster 2024-02-10T08:36:01.500350+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-10T08:36:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[24488]: audit 2024-02-10T08:36:01.546520+0000 mon.c (mon.2) 808 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[24488]: audit 2024-02-10T08:36:01.547296+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[24488]: audit 2024-02-10T08:36:01.548312+0000 mon.c (mon.2) 809 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-77"}]: dispatch 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[24488]: audit 2024-02-10T08:36:01.548912+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-77"}]: dispatch 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[17385]: audit 2024-02-10T08:36:01.500231+0000 mon.a (mon.0) 2344 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[17385]: cluster 2024-02-10T08:36:01.500350+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[17385]: audit 2024-02-10T08:36:01.546520+0000 mon.c (mon.2) 808 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[17385]: audit 2024-02-10T08:36:01.547296+0000 mon.a (mon.0) 2346 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[17385]: audit 2024-02-10T08:36:01.548312+0000 mon.c (mon.2) 809 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-77"}]: dispatch 2024-02-10T08:36:02.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:02 smithi067 bash[17385]: audit 2024-02-10T08:36:01.548912+0000 mon.a (mon.0) 2347 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-77"}]: dispatch 2024-02-10T08:36:03.509 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:36:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:36:03.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:03 smithi134 bash[18722]: cluster 2024-02-10T08:36:02.153657+0000 mgr.y (mgr.14547) 533 : cluster [DBG] pgmap v762: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-02-10T08:36:03.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:03 smithi134 bash[18722]: audit 2024-02-10T08:36:02.229798+0000 mgr.y (mgr.14547) 534 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:03.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:03 smithi134 bash[18722]: cluster 2024-02-10T08:36:02.509872+0000 mon.a (mon.0) 2348 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-10T08:36:03.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:03 smithi067 bash[24488]: cluster 2024-02-10T08:36:02.153657+0000 mgr.y (mgr.14547) 533 : cluster [DBG] pgmap v762: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-02-10T08:36:03.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:03 smithi067 bash[24488]: audit 2024-02-10T08:36:02.229798+0000 mgr.y (mgr.14547) 534 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:03.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:03 smithi067 bash[24488]: cluster 2024-02-10T08:36:02.509872+0000 mon.a (mon.0) 2348 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-10T08:36:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:03 smithi067 bash[17385]: cluster 2024-02-10T08:36:02.153657+0000 mgr.y (mgr.14547) 533 : cluster [DBG] pgmap v762: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 15 KiB/s wr, 40 op/s 2024-02-10T08:36:03.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:03 smithi067 bash[17385]: audit 2024-02-10T08:36:02.229798+0000 mgr.y (mgr.14547) 534 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:03.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:03 smithi067 bash[17385]: cluster 2024-02-10T08:36:02.509872+0000 mon.a (mon.0) 2348 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-10T08:36:04.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:04 smithi067 bash[17385]: cluster 2024-02-10T08:36:03.518987+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-10T08:36:04.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:04 smithi067 bash[17385]: audit 2024-02-10T08:36:03.521582+0000 mon.c (mon.2) 810 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:04.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:04 smithi067 bash[17385]: audit 2024-02-10T08:36:03.523508+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:04.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:04 smithi067 bash[24488]: cluster 2024-02-10T08:36:03.518987+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-10T08:36:04.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:04 smithi067 bash[24488]: audit 2024-02-10T08:36:03.521582+0000 mon.c (mon.2) 810 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:04.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:04 smithi067 bash[24488]: audit 2024-02-10T08:36:03.523508+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:05.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:04 smithi134 bash[18722]: cluster 2024-02-10T08:36:03.518987+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-10T08:36:05.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:04 smithi134 bash[18722]: audit 2024-02-10T08:36:03.521582+0000 mon.c (mon.2) 810 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:05.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:04 smithi134 bash[18722]: audit 2024-02-10T08:36:03.523508+0000 mon.a (mon.0) 2350 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[24488]: cluster 2024-02-10T08:36:04.154378+0000 mgr.y (mgr.14547) 535 : cluster [DBG] pgmap v765: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-10T08:36:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[24488]: audit 2024-02-10T08:36:04.531596+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[24488]: cluster 2024-02-10T08:36:04.531834+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-10T08:36:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[24488]: audit 2024-02-10T08:36:04.536222+0000 mon.c (mon.2) 811 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:05.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[24488]: audit 2024-02-10T08:36:04.540389+0000 mon.c (mon.2) 812 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:05.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[24488]: audit 2024-02-10T08:36:04.541950+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:05.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[17385]: cluster 2024-02-10T08:36:04.154378+0000 mgr.y (mgr.14547) 535 : cluster [DBG] pgmap v765: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-10T08:36:05.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[17385]: audit 2024-02-10T08:36:04.531596+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:05.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[17385]: cluster 2024-02-10T08:36:04.531834+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-10T08:36:05.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[17385]: audit 2024-02-10T08:36:04.536222+0000 mon.c (mon.2) 811 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:05.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[17385]: audit 2024-02-10T08:36:04.540389+0000 mon.c (mon.2) 812 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:05.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:05 smithi067 bash[17385]: audit 2024-02-10T08:36:04.541950+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:05 smithi134 bash[18722]: cluster 2024-02-10T08:36:04.154378+0000 mgr.y (mgr.14547) 535 : cluster [DBG] pgmap v765: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-10T08:36:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:05 smithi134 bash[18722]: audit 2024-02-10T08:36:04.531596+0000 mon.a (mon.0) 2351 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:05 smithi134 bash[18722]: cluster 2024-02-10T08:36:04.531834+0000 mon.a (mon.0) 2352 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-10T08:36:06.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:05 smithi134 bash[18722]: audit 2024-02-10T08:36:04.536222+0000 mon.c (mon.2) 811 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:06.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:05 smithi134 bash[18722]: audit 2024-02-10T08:36:04.540389+0000 mon.c (mon.2) 812 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:06.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:05 smithi134 bash[18722]: audit 2024-02-10T08:36:04.541950+0000 mon.a (mon.0) 2353 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[24488]: audit 2024-02-10T08:36:05.536550+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[24488]: cluster 2024-02-10T08:36:05.536686+0000 mon.a (mon.0) 2355 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-10T08:36:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[24488]: audit 2024-02-10T08:36:05.538061+0000 mon.c (mon.2) 813 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[24488]: audit 2024-02-10T08:36:05.539192+0000 mon.a (mon.0) 2356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:06.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[24488]: audit 2024-02-10T08:36:06.538923+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[24488]: cluster 2024-02-10T08:36:06.539136+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[17385]: audit 2024-02-10T08:36:05.536550+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[17385]: cluster 2024-02-10T08:36:05.536686+0000 mon.a (mon.0) 2355 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[17385]: audit 2024-02-10T08:36:05.538061+0000 mon.c (mon.2) 813 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[17385]: audit 2024-02-10T08:36:05.539192+0000 mon.a (mon.0) 2356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[17385]: audit 2024-02-10T08:36:06.538923+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:06 smithi067 bash[17385]: cluster 2024-02-10T08:36:06.539136+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-10T08:36:07.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:06 smithi134 bash[18722]: audit 2024-02-10T08:36:05.536550+0000 mon.a (mon.0) 2354 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:07.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:06 smithi134 bash[18722]: cluster 2024-02-10T08:36:05.536686+0000 mon.a (mon.0) 2355 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-10T08:36:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:06 smithi134 bash[18722]: audit 2024-02-10T08:36:05.538061+0000 mon.c (mon.2) 813 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:06 smithi134 bash[18722]: audit 2024-02-10T08:36:05.539192+0000 mon.a (mon.0) 2356 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:06 smithi134 bash[18722]: audit 2024-02-10T08:36:06.538923+0000 mon.a (mon.0) 2357 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:06 smithi134 bash[18722]: cluster 2024-02-10T08:36:06.539136+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-10T08:36:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[24488]: cluster 2024-02-10T08:36:06.155440+0000 mgr.y (mgr.14547) 536 : cluster [DBG] pgmap v768: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[24488]: audit 2024-02-10T08:36:06.547148+0000 mon.c (mon.2) 814 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[24488]: audit 2024-02-10T08:36:06.551184+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[17385]: cluster 2024-02-10T08:36:06.155440+0000 mgr.y (mgr.14547) 536 : cluster [DBG] pgmap v768: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[17385]: audit 2024-02-10T08:36:06.547148+0000 mon.c (mon.2) 814 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[17385]: audit 2024-02-10T08:36:06.551184+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:07 smithi134 bash[18722]: cluster 2024-02-10T08:36:06.155440+0000 mgr.y (mgr.14547) 536 : cluster [DBG] pgmap v768: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:07 smithi134 bash[18722]: audit 2024-02-10T08:36:06.547148+0000 mon.c (mon.2) 814 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:07 smithi134 bash[18722]: audit 2024-02-10T08:36:06.551184+0000 mon.a (mon.0) 2359 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:36:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:07] "GET /metrics HTTP/1.1" 200 189038 "" "Prometheus/2.33.4" 2024-02-10T08:36:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[17385]: audit 2024-02-10T08:36:07.557179+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[17385]: cluster 2024-02-10T08:36:07.557295+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-10T08:36:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[17385]: audit 2024-02-10T08:36:07.560434+0000 mon.c (mon.2) 815 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[17385]: audit 2024-02-10T08:36:07.564578+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[24488]: audit 2024-02-10T08:36:07.557179+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[24488]: cluster 2024-02-10T08:36:07.557295+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-10T08:36:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[24488]: audit 2024-02-10T08:36:07.560434+0000 mon.c (mon.2) 815 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:08 smithi067 bash[24488]: audit 2024-02-10T08:36:07.564578+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:08 smithi134 bash[18722]: audit 2024-02-10T08:36:07.557179+0000 mon.a (mon.0) 2360 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:08 smithi134 bash[18722]: cluster 2024-02-10T08:36:07.557295+0000 mon.a (mon.0) 2361 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-10T08:36:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:08 smithi134 bash[18722]: audit 2024-02-10T08:36:07.560434+0000 mon.c (mon.2) 815 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:08 smithi134 bash[18722]: audit 2024-02-10T08:36:07.564578+0000 mon.a (mon.0) 2362 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:09 smithi067 bash[17385]: cluster 2024-02-10T08:36:08.156279+0000 mgr.y (mgr.14547) 537 : cluster [DBG] pgmap v771: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:09 smithi067 bash[17385]: audit 2024-02-10T08:36:08.578481+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:09 smithi067 bash[17385]: cluster 2024-02-10T08:36:08.578607+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-10T08:36:09.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:09 smithi067 bash[24488]: cluster 2024-02-10T08:36:08.156279+0000 mgr.y (mgr.14547) 537 : cluster [DBG] pgmap v771: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:09 smithi067 bash[24488]: audit 2024-02-10T08:36:08.578481+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:09 smithi067 bash[24488]: cluster 2024-02-10T08:36:08.578607+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-10T08:36:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:09 smithi134 bash[18722]: cluster 2024-02-10T08:36:08.156279+0000 mgr.y (mgr.14547) 537 : cluster [DBG] pgmap v771: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:09 smithi134 bash[18722]: audit 2024-02-10T08:36:08.578481+0000 mon.a (mon.0) 2363 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:09 smithi134 bash[18722]: cluster 2024-02-10T08:36:08.578607+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-10T08:36:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:10 smithi067 bash[17385]: cluster 2024-02-10T08:36:09.586563+0000 mon.a (mon.0) 2365 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-10T08:36:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:10 smithi067 bash[24488]: cluster 2024-02-10T08:36:09.586563+0000 mon.a (mon.0) 2365 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-10T08:36:11.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:10 smithi134 bash[18722]: cluster 2024-02-10T08:36:09.586563+0000 mon.a (mon.0) 2365 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-10T08:36:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[17385]: cluster 2024-02-10T08:36:10.157141+0000 mgr.y (mgr.14547) 538 : cluster [DBG] pgmap v774: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[17385]: cluster 2024-02-10T08:36:10.597129+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-10T08:36:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[17385]: audit 2024-02-10T08:36:10.655603+0000 mon.c (mon.2) 816 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[17385]: audit 2024-02-10T08:36:10.656308+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[17385]: audit 2024-02-10T08:36:10.657421+0000 mon.c (mon.2) 817 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-79"}]: dispatch 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[17385]: audit 2024-02-10T08:36:10.657928+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-79"}]: dispatch 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[24488]: cluster 2024-02-10T08:36:10.157141+0000 mgr.y (mgr.14547) 538 : cluster [DBG] pgmap v774: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[24488]: cluster 2024-02-10T08:36:10.597129+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[24488]: audit 2024-02-10T08:36:10.655603+0000 mon.c (mon.2) 816 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[24488]: audit 2024-02-10T08:36:10.656308+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[24488]: audit 2024-02-10T08:36:10.657421+0000 mon.c (mon.2) 817 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-79"}]: dispatch 2024-02-10T08:36:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:11 smithi067 bash[24488]: audit 2024-02-10T08:36:10.657928+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-79"}]: dispatch 2024-02-10T08:36:12.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:11 smithi134 bash[18722]: cluster 2024-02-10T08:36:10.157141+0000 mgr.y (mgr.14547) 538 : cluster [DBG] pgmap v774: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:11 smithi134 bash[18722]: cluster 2024-02-10T08:36:10.597129+0000 mon.a (mon.0) 2366 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-10T08:36:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:11 smithi134 bash[18722]: audit 2024-02-10T08:36:10.655603+0000 mon.c (mon.2) 816 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:11 smithi134 bash[18722]: audit 2024-02-10T08:36:10.656308+0000 mon.a (mon.0) 2367 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:11 smithi134 bash[18722]: audit 2024-02-10T08:36:10.657421+0000 mon.c (mon.2) 817 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-79"}]: dispatch 2024-02-10T08:36:12.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:11 smithi134 bash[18722]: audit 2024-02-10T08:36:10.657928+0000 mon.a (mon.0) 2368 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-79"}]: dispatch 2024-02-10T08:36:12.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:36:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:36:12.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:12 smithi067 bash[17385]: cluster 2024-02-10T08:36:11.610474+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-10T08:36:12.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:12 smithi067 bash[24488]: cluster 2024-02-10T08:36:11.610474+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-10T08:36:13.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:12 smithi134 bash[18722]: cluster 2024-02-10T08:36:11.610474+0000 mon.a (mon.0) 2369 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-10T08:36:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:36:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:36:14.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:13 smithi134 bash[18722]: cluster 2024-02-10T08:36:12.158968+0000 mgr.y (mgr.14547) 539 : cluster [DBG] pgmap v777: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-10T08:36:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:13 smithi134 bash[18722]: audit 2024-02-10T08:36:12.231323+0000 mgr.y (mgr.14547) 540 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:13 smithi134 bash[18722]: cluster 2024-02-10T08:36:12.621458+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-10T08:36:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:13 smithi134 bash[18722]: audit 2024-02-10T08:36:12.625619+0000 mon.c (mon.2) 818 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:13 smithi134 bash[18722]: audit 2024-02-10T08:36:12.627185+0000 mon.a (mon.0) 2371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[17385]: cluster 2024-02-10T08:36:12.158968+0000 mgr.y (mgr.14547) 539 : cluster [DBG] pgmap v777: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-10T08:36:14.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[17385]: audit 2024-02-10T08:36:12.231323+0000 mgr.y (mgr.14547) 540 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:14.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[17385]: cluster 2024-02-10T08:36:12.621458+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-10T08:36:14.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[17385]: audit 2024-02-10T08:36:12.625619+0000 mon.c (mon.2) 818 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:14.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[17385]: audit 2024-02-10T08:36:12.627185+0000 mon.a (mon.0) 2371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:14.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[24488]: cluster 2024-02-10T08:36:12.158968+0000 mgr.y (mgr.14547) 539 : cluster [DBG] pgmap v777: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-10T08:36:14.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[24488]: audit 2024-02-10T08:36:12.231323+0000 mgr.y (mgr.14547) 540 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:14.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[24488]: cluster 2024-02-10T08:36:12.621458+0000 mon.a (mon.0) 2370 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-10T08:36:14.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[24488]: audit 2024-02-10T08:36:12.625619+0000 mon.c (mon.2) 818 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:14.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:13 smithi067 bash[24488]: audit 2024-02-10T08:36:12.627185+0000 mon.a (mon.0) 2371 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:13.627358+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: cluster 2024-02-10T08:36:13.628040+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:13.632795+0000 mon.c (mon.2) 819 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:13.658166+0000 mon.c (mon.2) 820 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:13.658931+0000 mon.a (mon.0) 2374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:14.629466+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: cluster 2024-02-10T08:36:14.629787+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:14.634497+0000 mon.c (mon.2) 821 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:14 smithi134 bash[18722]: audit 2024-02-10T08:36:14.634997+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:15.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:13.627358+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:15.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: cluster 2024-02-10T08:36:13.628040+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-10T08:36:15.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:13.632795+0000 mon.c (mon.2) 819 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:15.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:13.658166+0000 mon.c (mon.2) 820 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:13.658931+0000 mon.a (mon.0) 2374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:14.629466+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: cluster 2024-02-10T08:36:14.629787+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:14.634497+0000 mon.c (mon.2) 821 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[24488]: audit 2024-02-10T08:36:14.634997+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:13.627358+0000 mon.a (mon.0) 2372 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: cluster 2024-02-10T08:36:13.628040+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:13.632795+0000 mon.c (mon.2) 819 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:13.658166+0000 mon.c (mon.2) 820 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:13.658931+0000 mon.a (mon.0) 2374 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:15.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:14.629466+0000 mon.a (mon.0) 2375 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:15.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: cluster 2024-02-10T08:36:14.629787+0000 mon.a (mon.0) 2376 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-10T08:36:15.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:14.634497+0000 mon.c (mon.2) 821 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:15.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:14 smithi067 bash[17385]: audit 2024-02-10T08:36:14.634997+0000 mon.a (mon.0) 2377 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:15 smithi134 bash[18722]: cluster 2024-02-10T08:36:14.159778+0000 mgr.y (mgr.14547) 541 : cluster [DBG] pgmap v780: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-10T08:36:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:15 smithi134 bash[18722]: audit 2024-02-10T08:36:15.633495+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:15 smithi134 bash[18722]: cluster 2024-02-10T08:36:15.633697+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-10T08:36:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:15 smithi134 bash[18722]: audit 2024-02-10T08:36:15.635620+0000 mon.c (mon.2) 822 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:15 smithi134 bash[18722]: audit 2024-02-10T08:36:15.636542+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[24488]: cluster 2024-02-10T08:36:14.159778+0000 mgr.y (mgr.14547) 541 : cluster [DBG] pgmap v780: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-10T08:36:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[24488]: audit 2024-02-10T08:36:15.633495+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[24488]: cluster 2024-02-10T08:36:15.633697+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-10T08:36:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[24488]: audit 2024-02-10T08:36:15.635620+0000 mon.c (mon.2) 822 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:16.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[24488]: audit 2024-02-10T08:36:15.636542+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[17385]: cluster 2024-02-10T08:36:14.159778+0000 mgr.y (mgr.14547) 541 : cluster [DBG] pgmap v780: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-10T08:36:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[17385]: audit 2024-02-10T08:36:15.633495+0000 mon.a (mon.0) 2378 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_tier","val": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[17385]: cluster 2024-02-10T08:36:15.633697+0000 mon.a (mon.0) 2379 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-10T08:36:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[17385]: audit 2024-02-10T08:36:15.635620+0000 mon.c (mon.2) 822 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:16.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:15 smithi067 bash[17385]: audit 2024-02-10T08:36:15.636542+0000 mon.a (mon.0) 2380 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:18.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:17 smithi134 bash[18722]: cluster 2024-02-10T08:36:16.160616+0000 mgr.y (mgr.14547) 542 : cluster [DBG] pgmap v783: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:17 smithi134 bash[18722]: audit 2024-02-10T08:36:16.635472+0000 mon.a (mon.0) 2381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:17 smithi134 bash[18722]: cluster 2024-02-10T08:36:16.635731+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-10T08:36:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:17 smithi134 bash[18722]: audit 2024-02-10T08:36:16.638935+0000 mon.c (mon.2) 823 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:18.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:17 smithi134 bash[18722]: audit 2024-02-10T08:36:16.639532+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:17] "GET /metrics HTTP/1.1" 200 189166 "" "Prometheus/2.33.4" 2024-02-10T08:36:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[17385]: cluster 2024-02-10T08:36:16.160616+0000 mgr.y (mgr.14547) 542 : cluster [DBG] pgmap v783: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[17385]: audit 2024-02-10T08:36:16.635472+0000 mon.a (mon.0) 2381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[17385]: cluster 2024-02-10T08:36:16.635731+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[17385]: audit 2024-02-10T08:36:16.638935+0000 mon.c (mon.2) 823 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[17385]: audit 2024-02-10T08:36:16.639532+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[24488]: cluster 2024-02-10T08:36:16.160616+0000 mgr.y (mgr.14547) 542 : cluster [DBG] pgmap v783: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[24488]: audit 2024-02-10T08:36:16.635472+0000 mon.a (mon.0) 2381 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[24488]: cluster 2024-02-10T08:36:16.635731+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[24488]: audit 2024-02-10T08:36:16.638935+0000 mon.c (mon.2) 823 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:17 smithi067 bash[24488]: audit 2024-02-10T08:36:16.639532+0000 mon.a (mon.0) 2383 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:19.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:18 smithi134 bash[18722]: audit 2024-02-10T08:36:17.643696+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:19.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:18 smithi134 bash[18722]: cluster 2024-02-10T08:36:17.643812+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-10T08:36:19.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:18 smithi067 bash[17385]: audit 2024-02-10T08:36:17.643696+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:19.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:18 smithi067 bash[17385]: cluster 2024-02-10T08:36:17.643812+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-10T08:36:19.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:18 smithi067 bash[24488]: audit 2024-02-10T08:36:17.643696+0000 mon.a (mon.0) 2384 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:19.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:18 smithi067 bash[24488]: cluster 2024-02-10T08:36:17.643812+0000 mon.a (mon.0) 2385 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-10T08:36:20.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:19 smithi134 bash[18722]: cluster 2024-02-10T08:36:18.161498+0000 mgr.y (mgr.14547) 543 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:20.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:19 smithi134 bash[18722]: cluster 2024-02-10T08:36:18.655599+0000 mon.a (mon.0) 2386 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-10T08:36:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:19 smithi067 bash[17385]: cluster 2024-02-10T08:36:18.161498+0000 mgr.y (mgr.14547) 543 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:19 smithi067 bash[17385]: cluster 2024-02-10T08:36:18.655599+0000 mon.a (mon.0) 2386 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-10T08:36:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:19 smithi067 bash[24488]: cluster 2024-02-10T08:36:18.161498+0000 mgr.y (mgr.14547) 543 : cluster [DBG] pgmap v786: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:19 smithi067 bash[24488]: cluster 2024-02-10T08:36:18.655599+0000 mon.a (mon.0) 2386 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-10T08:36:21.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:20 smithi134 bash[18722]: cluster 2024-02-10T08:36:19.660541+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-10T08:36:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:20 smithi134 bash[18722]: audit 2024-02-10T08:36:19.715411+0000 mon.c (mon.2) 824 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:20 smithi134 bash[18722]: audit 2024-02-10T08:36:19.716171+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:20 smithi134 bash[18722]: audit 2024-02-10T08:36:19.717273+0000 mon.c (mon.2) 825 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-81"}]: dispatch 2024-02-10T08:36:21.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:20 smithi134 bash[18722]: audit 2024-02-10T08:36:19.717867+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-81"}]: dispatch 2024-02-10T08:36:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[17385]: cluster 2024-02-10T08:36:19.660541+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-10T08:36:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[17385]: audit 2024-02-10T08:36:19.715411+0000 mon.c (mon.2) 824 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[17385]: audit 2024-02-10T08:36:19.716171+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[17385]: audit 2024-02-10T08:36:19.717273+0000 mon.c (mon.2) 825 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-81"}]: dispatch 2024-02-10T08:36:21.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[17385]: audit 2024-02-10T08:36:19.717867+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-81"}]: dispatch 2024-02-10T08:36:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[24488]: cluster 2024-02-10T08:36:19.660541+0000 mon.a (mon.0) 2387 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-10T08:36:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[24488]: audit 2024-02-10T08:36:19.715411+0000 mon.c (mon.2) 824 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[24488]: audit 2024-02-10T08:36:19.716171+0000 mon.a (mon.0) 2388 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[24488]: audit 2024-02-10T08:36:19.717273+0000 mon.c (mon.2) 825 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-81"}]: dispatch 2024-02-10T08:36:21.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:20 smithi067 bash[24488]: audit 2024-02-10T08:36:19.717867+0000 mon.a (mon.0) 2389 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-81"}]: dispatch 2024-02-10T08:36:22.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:21 smithi134 bash[18722]: cluster 2024-02-10T08:36:20.162241+0000 mgr.y (mgr.14547) 544 : cluster [DBG] pgmap v789: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:22.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:21 smithi134 bash[18722]: cluster 2024-02-10T08:36:20.680168+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-10T08:36:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:21 smithi067 bash[17385]: cluster 2024-02-10T08:36:20.162241+0000 mgr.y (mgr.14547) 544 : cluster [DBG] pgmap v789: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:21 smithi067 bash[17385]: cluster 2024-02-10T08:36:20.680168+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-10T08:36:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:21 smithi067 bash[24488]: cluster 2024-02-10T08:36:20.162241+0000 mgr.y (mgr.14547) 544 : cluster [DBG] pgmap v789: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:21 smithi067 bash[24488]: cluster 2024-02-10T08:36:20.680168+0000 mon.a (mon.0) 2390 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-10T08:36:22.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:36:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:36:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:36:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[18722]: cluster 2024-02-10T08:36:21.684798+0000 mon.a (mon.0) 2391 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-10T08:36:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[18722]: audit 2024-02-10T08:36:21.688019+0000 mon.c (mon.2) 826 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[18722]: audit 2024-02-10T08:36:21.690029+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[18722]: audit 2024-02-10T08:36:22.687267+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[18722]: cluster 2024-02-10T08:36:22.687669+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-10T08:36:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:23 smithi134 bash[18722]: audit 2024-02-10T08:36:22.691998+0000 mon.c (mon.2) 827 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[17385]: cluster 2024-02-10T08:36:21.684798+0000 mon.a (mon.0) 2391 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-10T08:36:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[17385]: audit 2024-02-10T08:36:21.688019+0000 mon.c (mon.2) 826 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[17385]: audit 2024-02-10T08:36:21.690029+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[17385]: audit 2024-02-10T08:36:22.687267+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[17385]: cluster 2024-02-10T08:36:22.687669+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[17385]: audit 2024-02-10T08:36:22.691998+0000 mon.c (mon.2) 827 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[24488]: cluster 2024-02-10T08:36:21.684798+0000 mon.a (mon.0) 2391 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[24488]: audit 2024-02-10T08:36:21.688019+0000 mon.c (mon.2) 826 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[24488]: audit 2024-02-10T08:36:21.690029+0000 mon.a (mon.0) 2392 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[24488]: audit 2024-02-10T08:36:22.687267+0000 mon.a (mon.0) 2393 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[24488]: cluster 2024-02-10T08:36:22.687669+0000 mon.a (mon.0) 2394 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-10T08:36:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:23 smithi067 bash[24488]: audit 2024-02-10T08:36:22.691998+0000 mon.c (mon.2) 827 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:24.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:24 smithi134 bash[18722]: cluster 2024-02-10T08:36:22.163134+0000 mgr.y (mgr.14547) 545 : cluster [DBG] pgmap v792: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T08:36:24.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:24 smithi134 bash[18722]: audit 2024-02-10T08:36:22.234534+0000 mgr.y (mgr.14547) 546 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:24 smithi134 bash[18722]: cluster 2024-02-10T08:36:23.690965+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-10T08:36:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:24 smithi134 bash[18722]: audit 2024-02-10T08:36:23.694648+0000 mon.c (mon.2) 828 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:24.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:24 smithi134 bash[18722]: audit 2024-02-10T08:36:23.698910+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[17385]: cluster 2024-02-10T08:36:22.163134+0000 mgr.y (mgr.14547) 545 : cluster [DBG] pgmap v792: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T08:36:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[17385]: audit 2024-02-10T08:36:22.234534+0000 mgr.y (mgr.14547) 546 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[17385]: cluster 2024-02-10T08:36:23.690965+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-10T08:36:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[17385]: audit 2024-02-10T08:36:23.694648+0000 mon.c (mon.2) 828 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:24.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[17385]: audit 2024-02-10T08:36:23.698910+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[24488]: cluster 2024-02-10T08:36:22.163134+0000 mgr.y (mgr.14547) 545 : cluster [DBG] pgmap v792: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T08:36:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[24488]: audit 2024-02-10T08:36:22.234534+0000 mgr.y (mgr.14547) 546 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[24488]: cluster 2024-02-10T08:36:23.690965+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-10T08:36:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[24488]: audit 2024-02-10T08:36:23.694648+0000 mon.c (mon.2) 828 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:24.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:24 smithi067 bash[24488]: audit 2024-02-10T08:36:23.698910+0000 mon.a (mon.0) 2396 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:26.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:25 smithi134 bash[18722]: cluster 2024-02-10T08:36:24.163731+0000 mgr.y (mgr.14547) 547 : cluster [DBG] pgmap v795: 260 pgs: 64 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T08:36:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:25 smithi134 bash[18722]: audit 2024-02-10T08:36:24.695382+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:25 smithi134 bash[18722]: cluster 2024-02-10T08:36:24.695811+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-10T08:36:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:25 smithi134 bash[18722]: audit 2024-02-10T08:36:24.697325+0000 mon.c (mon.2) 829 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]: dispatch 2024-02-10T08:36:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:25 smithi134 bash[18722]: audit 2024-02-10T08:36:24.699418+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]: dispatch 2024-02-10T08:36:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[17385]: cluster 2024-02-10T08:36:24.163731+0000 mgr.y (mgr.14547) 547 : cluster [DBG] pgmap v795: 260 pgs: 64 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T08:36:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[17385]: audit 2024-02-10T08:36:24.695382+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[17385]: cluster 2024-02-10T08:36:24.695811+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[17385]: audit 2024-02-10T08:36:24.697325+0000 mon.c (mon.2) 829 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]: dispatch 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[17385]: audit 2024-02-10T08:36:24.699418+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]: dispatch 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[24488]: cluster 2024-02-10T08:36:24.163731+0000 mgr.y (mgr.14547) 547 : cluster [DBG] pgmap v795: 260 pgs: 64 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[24488]: audit 2024-02-10T08:36:24.695382+0000 mon.a (mon.0) 2397 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[24488]: cluster 2024-02-10T08:36:24.695811+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[24488]: audit 2024-02-10T08:36:24.697325+0000 mon.c (mon.2) 829 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]: dispatch 2024-02-10T08:36:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:25 smithi067 bash[24488]: audit 2024-02-10T08:36:24.699418+0000 mon.a (mon.0) 2399 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]: dispatch 2024-02-10T08:36:27.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:26 smithi134 bash[18722]: audit 2024-02-10T08:36:25.699000+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]': finished 2024-02-10T08:36:27.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:26 smithi134 bash[18722]: cluster 2024-02-10T08:36:25.699185+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-10T08:36:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:26 smithi134 bash[18722]: audit 2024-02-10T08:36:25.700915+0000 mon.c (mon.2) 830 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:26 smithi134 bash[18722]: audit 2024-02-10T08:36:25.701578+0000 mon.a (mon.0) 2402 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:26 smithi134 bash[18722]: audit 2024-02-10T08:36:26.699309+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:27.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:26 smithi134 bash[18722]: cluster 2024-02-10T08:36:26.699555+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-10T08:36:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[17385]: audit 2024-02-10T08:36:25.699000+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]': finished 2024-02-10T08:36:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[17385]: cluster 2024-02-10T08:36:25.699185+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-10T08:36:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[17385]: audit 2024-02-10T08:36:25.700915+0000 mon.c (mon.2) 830 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[17385]: audit 2024-02-10T08:36:25.701578+0000 mon.a (mon.0) 2402 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[17385]: audit 2024-02-10T08:36:26.699309+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:27.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[17385]: cluster 2024-02-10T08:36:26.699555+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-10T08:36:27.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[24488]: audit 2024-02-10T08:36:25.699000+0000 mon.a (mon.0) 2400 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_tier","val": "test-rados-api-smithi067-54339-85-test-flush"}]': finished 2024-02-10T08:36:27.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[24488]: cluster 2024-02-10T08:36:25.699185+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-10T08:36:27.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[24488]: audit 2024-02-10T08:36:25.700915+0000 mon.c (mon.2) 830 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:27.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[24488]: audit 2024-02-10T08:36:25.701578+0000 mon.a (mon.0) 2402 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:36:27.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[24488]: audit 2024-02-10T08:36:26.699309+0000 mon.a (mon.0) 2403 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:36:27.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:26 smithi067 bash[24488]: cluster 2024-02-10T08:36:26.699555+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-10T08:36:28.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:27 smithi134 bash[18722]: cluster 2024-02-10T08:36:26.164590+0000 mgr.y (mgr.14547) 548 : cluster [DBG] pgmap v798: 260 pgs: 64 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:28.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:27 smithi134 bash[18722]: audit 2024-02-10T08:36:26.701776+0000 mon.c (mon.2) 831 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:28.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:27 smithi134 bash[18722]: audit 2024-02-10T08:36:26.702243+0000 mon.a (mon.0) 2405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:28.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:27 smithi134 bash[18722]: audit 2024-02-10T08:36:27.704347+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:28.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:27 smithi134 bash[18722]: cluster 2024-02-10T08:36:27.704446+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-10T08:36:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:27] "GET /metrics HTTP/1.1" 200 189166 "" "Prometheus/2.33.4" 2024-02-10T08:36:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[17385]: cluster 2024-02-10T08:36:26.164590+0000 mgr.y (mgr.14547) 548 : cluster [DBG] pgmap v798: 260 pgs: 64 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[17385]: audit 2024-02-10T08:36:26.701776+0000 mon.c (mon.2) 831 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[17385]: audit 2024-02-10T08:36:26.702243+0000 mon.a (mon.0) 2405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[17385]: audit 2024-02-10T08:36:27.704347+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:28.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[17385]: cluster 2024-02-10T08:36:27.704446+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-10T08:36:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[24488]: cluster 2024-02-10T08:36:26.164590+0000 mgr.y (mgr.14547) 548 : cluster [DBG] pgmap v798: 260 pgs: 64 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[24488]: audit 2024-02-10T08:36:26.701776+0000 mon.c (mon.2) 831 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[24488]: audit 2024-02-10T08:36:26.702243+0000 mon.a (mon.0) 2405 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:36:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[24488]: audit 2024-02-10T08:36:27.704347+0000 mon.a (mon.0) 2406 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:36:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:27 smithi067 bash[24488]: cluster 2024-02-10T08:36:27.704446+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-10T08:36:30.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:29 smithi134 bash[18722]: cluster 2024-02-10T08:36:28.165483+0000 mgr.y (mgr.14547) 549 : cluster [DBG] pgmap v801: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:29 smithi134 bash[18722]: cluster 2024-02-10T08:36:28.719685+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-10T08:36:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:29 smithi134 bash[18722]: audit 2024-02-10T08:36:28.777370+0000 mon.c (mon.2) 832 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:29 smithi134 bash[18722]: audit 2024-02-10T08:36:28.777991+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:29 smithi134 bash[18722]: audit 2024-02-10T08:36:28.779088+0000 mon.c (mon.2) 833 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-83"}]: dispatch 2024-02-10T08:36:30.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:29 smithi134 bash[18722]: audit 2024-02-10T08:36:28.779590+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-83"}]: dispatch 2024-02-10T08:36:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[17385]: cluster 2024-02-10T08:36:28.165483+0000 mgr.y (mgr.14547) 549 : cluster [DBG] pgmap v801: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[17385]: cluster 2024-02-10T08:36:28.719685+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-10T08:36:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[17385]: audit 2024-02-10T08:36:28.777370+0000 mon.c (mon.2) 832 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[17385]: audit 2024-02-10T08:36:28.777991+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[17385]: audit 2024-02-10T08:36:28.779088+0000 mon.c (mon.2) 833 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-83"}]: dispatch 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[17385]: audit 2024-02-10T08:36:28.779590+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-83"}]: dispatch 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[24488]: cluster 2024-02-10T08:36:28.165483+0000 mgr.y (mgr.14547) 549 : cluster [DBG] pgmap v801: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[24488]: cluster 2024-02-10T08:36:28.719685+0000 mon.a (mon.0) 2408 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[24488]: audit 2024-02-10T08:36:28.777370+0000 mon.c (mon.2) 832 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[24488]: audit 2024-02-10T08:36:28.777991+0000 mon.a (mon.0) 2409 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[24488]: audit 2024-02-10T08:36:28.779088+0000 mon.c (mon.2) 833 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-83"}]: dispatch 2024-02-10T08:36:30.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:29 smithi067 bash[24488]: audit 2024-02-10T08:36:28.779590+0000 mon.a (mon.0) 2410 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-83"}]: dispatch 2024-02-10T08:36:31.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:30 smithi134 bash[18722]: cluster 2024-02-10T08:36:29.732828+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-10T08:36:31.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:30 smithi067 bash[17385]: cluster 2024-02-10T08:36:29.732828+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-10T08:36:31.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:30 smithi067 bash[24488]: cluster 2024-02-10T08:36:29.732828+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-10T08:36:32.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:31 smithi134 bash[18722]: cluster 2024-02-10T08:36:30.166247+0000 mgr.y (mgr.14547) 550 : cluster [DBG] pgmap v804: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:31 smithi134 bash[18722]: cluster 2024-02-10T08:36:30.749471+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-10T08:36:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:31 smithi134 bash[18722]: audit 2024-02-10T08:36:30.751531+0000 mon.c (mon.2) 834 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:32.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:31 smithi134 bash[18722]: audit 2024-02-10T08:36:30.752123+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[24488]: cluster 2024-02-10T08:36:30.166247+0000 mgr.y (mgr.14547) 550 : cluster [DBG] pgmap v804: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[24488]: cluster 2024-02-10T08:36:30.749471+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-10T08:36:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[24488]: audit 2024-02-10T08:36:30.751531+0000 mon.c (mon.2) 834 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[24488]: audit 2024-02-10T08:36:30.752123+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[17385]: cluster 2024-02-10T08:36:30.166247+0000 mgr.y (mgr.14547) 550 : cluster [DBG] pgmap v804: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[17385]: cluster 2024-02-10T08:36:30.749471+0000 mon.a (mon.0) 2412 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-10T08:36:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[17385]: audit 2024-02-10T08:36:30.751531+0000 mon.c (mon.2) 834 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:31 smithi067 bash[17385]: audit 2024-02-10T08:36:30.752123+0000 mon.a (mon.0) 2413 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:32.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:36:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:36:33.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:32 smithi134 bash[18722]: audit 2024-02-10T08:36:31.750531+0000 mon.a (mon.0) 2414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:33.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:32 smithi134 bash[18722]: cluster 2024-02-10T08:36:31.751093+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-10T08:36:33.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:32 smithi134 bash[18722]: audit 2024-02-10T08:36:31.757876+0000 mon.c (mon.2) 835 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:33.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:32 smithi134 bash[18722]: audit 2024-02-10T08:36:31.772055+0000 mon.c (mon.2) 836 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:33.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:32 smithi134 bash[18722]: audit 2024-02-10T08:36:31.773318+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:33.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[24488]: audit 2024-02-10T08:36:31.750531+0000 mon.a (mon.0) 2414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:33.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[24488]: cluster 2024-02-10T08:36:31.751093+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-10T08:36:33.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[24488]: audit 2024-02-10T08:36:31.757876+0000 mon.c (mon.2) 835 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[24488]: audit 2024-02-10T08:36:31.772055+0000 mon.c (mon.2) 836 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[24488]: audit 2024-02-10T08:36:31.773318+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[17385]: audit 2024-02-10T08:36:31.750531+0000 mon.a (mon.0) 2414 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[17385]: cluster 2024-02-10T08:36:31.751093+0000 mon.a (mon.0) 2415 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[17385]: audit 2024-02-10T08:36:31.757876+0000 mon.c (mon.2) 835 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[17385]: audit 2024-02-10T08:36:31.772055+0000 mon.c (mon.2) 836 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:33.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:32 smithi067 bash[17385]: audit 2024-02-10T08:36:31.773318+0000 mon.a (mon.0) 2416 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:36:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:36:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:36:34.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[24488]: cluster 2024-02-10T08:36:32.167104+0000 mgr.y (mgr.14547) 551 : cluster [DBG] pgmap v807: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:36:34.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[24488]: audit 2024-02-10T08:36:32.246025+0000 mgr.y (mgr.14547) 552 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:34.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[24488]: audit 2024-02-10T08:36:32.774468+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:34.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[24488]: cluster 2024-02-10T08:36:32.774573+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-10T08:36:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[17385]: cluster 2024-02-10T08:36:32.167104+0000 mgr.y (mgr.14547) 551 : cluster [DBG] pgmap v807: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:36:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[17385]: audit 2024-02-10T08:36:32.246025+0000 mgr.y (mgr.14547) 552 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[17385]: audit 2024-02-10T08:36:32.774468+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:34.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:33 smithi067 bash[17385]: cluster 2024-02-10T08:36:32.774573+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-10T08:36:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:33 smithi134 bash[18722]: cluster 2024-02-10T08:36:32.167104+0000 mgr.y (mgr.14547) 551 : cluster [DBG] pgmap v807: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:36:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:33 smithi134 bash[18722]: audit 2024-02-10T08:36:32.246025+0000 mgr.y (mgr.14547) 552 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:33 smithi134 bash[18722]: audit 2024-02-10T08:36:32.774468+0000 mon.a (mon.0) 2417 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:36:34.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:33 smithi134 bash[18722]: cluster 2024-02-10T08:36:32.774573+0000 mon.a (mon.0) 2418 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-10T08:36:35.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:34 smithi067 bash[24488]: cluster 2024-02-10T08:36:33.779066+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-10T08:36:35.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:34 smithi067 bash[17385]: cluster 2024-02-10T08:36:33.779066+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-10T08:36:35.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:34 smithi134 bash[18722]: cluster 2024-02-10T08:36:33.779066+0000 mon.a (mon.0) 2419 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22170 ms) 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3020 ms) 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3019 ms) 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3029 ms) 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.814 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3045 ms) 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4062 ms) 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39345 ms) 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17101 ms) 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4055 ms) 2024-02-10T08:36:35.815 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3030 ms) 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24824 ms) 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15068 ms) 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5056 ms) 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.816 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: waiting for scrubs... 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: done waiting 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25136 ms) 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10109 ms) 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9096 ms) 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9072 ms) 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.817 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9054 ms) 2024-02-10T08:36:35.818 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-10T08:36:35.818 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:36:35.818 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6073 ms) 2024-02-10T08:36:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[17385]: cluster 2024-02-10T08:36:34.167894+0000 mgr.y (mgr.14547) 553 : cluster [DBG] pgmap v810: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:36:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[17385]: cluster 2024-02-10T08:36:34.790234+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-10T08:36:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[17385]: audit 2024-02-10T08:36:34.856581+0000 mon.c (mon.2) 837 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[17385]: audit 2024-02-10T08:36:34.857285+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[17385]: audit 2024-02-10T08:36:34.858389+0000 mon.c (mon.2) 838 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-86"}]: dispatch 2024-02-10T08:36:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[17385]: audit 2024-02-10T08:36:34.859036+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-86"}]: dispatch 2024-02-10T08:36:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[24488]: cluster 2024-02-10T08:36:34.167894+0000 mgr.y (mgr.14547) 553 : cluster [DBG] pgmap v810: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:36:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[24488]: cluster 2024-02-10T08:36:34.790234+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-10T08:36:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[24488]: audit 2024-02-10T08:36:34.856581+0000 mon.c (mon.2) 837 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[24488]: audit 2024-02-10T08:36:34.857285+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[24488]: audit 2024-02-10T08:36:34.858389+0000 mon.c (mon.2) 838 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-86"}]: dispatch 2024-02-10T08:36:36.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:35 smithi067 bash[24488]: audit 2024-02-10T08:36:34.859036+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-86"}]: dispatch 2024-02-10T08:36:36.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:35 smithi134 bash[18722]: cluster 2024-02-10T08:36:34.167894+0000 mgr.y (mgr.14547) 553 : cluster [DBG] pgmap v810: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:36:36.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:35 smithi134 bash[18722]: cluster 2024-02-10T08:36:34.790234+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-10T08:36:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:35 smithi134 bash[18722]: audit 2024-02-10T08:36:34.856581+0000 mon.c (mon.2) 837 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:35 smithi134 bash[18722]: audit 2024-02-10T08:36:34.857285+0000 mon.a (mon.0) 2421 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:35 smithi134 bash[18722]: audit 2024-02-10T08:36:34.858389+0000 mon.c (mon.2) 838 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-86"}]: dispatch 2024-02-10T08:36:36.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:35 smithi134 bash[18722]: audit 2024-02-10T08:36:34.859036+0000 mon.a (mon.0) 2422 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-86"}]: dispatch 2024-02-10T08:36:37.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:36 smithi067 bash[24488]: cluster 2024-02-10T08:36:35.810994+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-10T08:36:37.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:36 smithi067 bash[17385]: cluster 2024-02-10T08:36:35.810994+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-10T08:36:37.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:36 smithi134 bash[18722]: cluster 2024-02-10T08:36:35.810994+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-10T08:36:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:37] "GET /metrics HTTP/1.1" 200 189295 "" "Prometheus/2.33.4" 2024-02-10T08:36:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[24488]: cluster 2024-02-10T08:36:36.168602+0000 mgr.y (mgr.14547) 554 : cluster [DBG] pgmap v813: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[24488]: cluster 2024-02-10T08:36:36.821729+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[24488]: audit 2024-02-10T08:36:36.829451+0000 mon.c (mon.2) 839 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[24488]: audit 2024-02-10T08:36:36.834838+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[17385]: cluster 2024-02-10T08:36:36.168602+0000 mgr.y (mgr.14547) 554 : cluster [DBG] pgmap v813: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[17385]: cluster 2024-02-10T08:36:36.821729+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[17385]: audit 2024-02-10T08:36:36.829451+0000 mon.c (mon.2) 839 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:37 smithi067 bash[17385]: audit 2024-02-10T08:36:36.834838+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:37 smithi134 bash[18722]: cluster 2024-02-10T08:36:36.168602+0000 mgr.y (mgr.14547) 554 : cluster [DBG] pgmap v813: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:37 smithi134 bash[18722]: cluster 2024-02-10T08:36:36.821729+0000 mon.a (mon.0) 2424 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-10T08:36:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:37 smithi134 bash[18722]: audit 2024-02-10T08:36:36.829451+0000 mon.c (mon.2) 839 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:37 smithi134 bash[18722]: audit 2024-02-10T08:36:36.834838+0000 mon.a (mon.0) 2425 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:39.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[17385]: audit 2024-02-10T08:36:37.834950+0000 mon.a (mon.0) 2426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:39.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[17385]: cluster 2024-02-10T08:36:37.835169+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-10T08:36:39.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[17385]: audit 2024-02-10T08:36:37.860873+0000 mon.c (mon.2) 840 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:39.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[17385]: audit 2024-02-10T08:36:37.861354+0000 mon.a (mon.0) 2428 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[24488]: audit 2024-02-10T08:36:37.834950+0000 mon.a (mon.0) 2426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[24488]: cluster 2024-02-10T08:36:37.835169+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-10T08:36:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[24488]: audit 2024-02-10T08:36:37.860873+0000 mon.c (mon.2) 840 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:39.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:38 smithi067 bash[24488]: audit 2024-02-10T08:36:37.861354+0000 mon.a (mon.0) 2428 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:39.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:38 smithi134 bash[18722]: audit 2024-02-10T08:36:37.834950+0000 mon.a (mon.0) 2426 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:39.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:38 smithi134 bash[18722]: cluster 2024-02-10T08:36:37.835169+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-10T08:36:39.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:38 smithi134 bash[18722]: audit 2024-02-10T08:36:37.860873+0000 mon.c (mon.2) 840 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:39.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:38 smithi134 bash[18722]: audit 2024-02-10T08:36:37.861354+0000 mon.a (mon.0) 2428 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[17385]: cluster 2024-02-10T08:36:38.169407+0000 mgr.y (mgr.14547) 555 : cluster [DBG] pgmap v816: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:36:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[17385]: audit 2024-02-10T08:36:38.839176+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:36:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[17385]: cluster 2024-02-10T08:36:38.839307+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-10T08:36:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[17385]: audit 2024-02-10T08:36:38.842155+0000 mon.c (mon.2) 841 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]: dispatch 2024-02-10T08:36:40.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[17385]: audit 2024-02-10T08:36:38.847134+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]: dispatch 2024-02-10T08:36:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[24488]: cluster 2024-02-10T08:36:38.169407+0000 mgr.y (mgr.14547) 555 : cluster [DBG] pgmap v816: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:36:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[24488]: audit 2024-02-10T08:36:38.839176+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:36:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[24488]: cluster 2024-02-10T08:36:38.839307+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-10T08:36:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[24488]: audit 2024-02-10T08:36:38.842155+0000 mon.c (mon.2) 841 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]: dispatch 2024-02-10T08:36:40.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:39 smithi067 bash[24488]: audit 2024-02-10T08:36:38.847134+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]: dispatch 2024-02-10T08:36:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:39 smithi134 bash[18722]: cluster 2024-02-10T08:36:38.169407+0000 mgr.y (mgr.14547) 555 : cluster [DBG] pgmap v816: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:36:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:39 smithi134 bash[18722]: audit 2024-02-10T08:36:38.839176+0000 mon.a (mon.0) 2429 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:36:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:39 smithi134 bash[18722]: cluster 2024-02-10T08:36:38.839307+0000 mon.a (mon.0) 2430 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-10T08:36:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:39 smithi134 bash[18722]: audit 2024-02-10T08:36:38.842155+0000 mon.c (mon.2) 841 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]: dispatch 2024-02-10T08:36:40.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:39 smithi134 bash[18722]: audit 2024-02-10T08:36:38.847134+0000 mon.a (mon.0) 2431 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]: dispatch 2024-02-10T08:36:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[17385]: cluster 2024-02-10T08:36:39.844049+0000 mon.a (mon.0) 2432 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:36:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[17385]: audit 2024-02-10T08:36:39.856147+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]': finished 2024-02-10T08:36:41.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[17385]: cluster 2024-02-10T08:36:39.856267+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[17385]: audit 2024-02-10T08:36:39.861105+0000 mon.c (mon.2) 842 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[17385]: audit 2024-02-10T08:36:39.861607+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[24488]: cluster 2024-02-10T08:36:39.844049+0000 mon.a (mon.0) 2432 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[24488]: audit 2024-02-10T08:36:39.856147+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]': finished 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[24488]: cluster 2024-02-10T08:36:39.856267+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[24488]: audit 2024-02-10T08:36:39.861105+0000 mon.c (mon.2) 842 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:41.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:40 smithi067 bash[24488]: audit 2024-02-10T08:36:39.861607+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:40 smithi134 bash[18722]: cluster 2024-02-10T08:36:39.844049+0000 mon.a (mon.0) 2432 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:36:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:40 smithi134 bash[18722]: audit 2024-02-10T08:36:39.856147+0000 mon.a (mon.0) 2433 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-88", "mode": "writeback"}]': finished 2024-02-10T08:36:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:40 smithi134 bash[18722]: cluster 2024-02-10T08:36:39.856267+0000 mon.a (mon.0) 2434 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-10T08:36:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:40 smithi134 bash[18722]: audit 2024-02-10T08:36:39.861105+0000 mon.c (mon.2) 842 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:41.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:40 smithi134 bash[18722]: audit 2024-02-10T08:36:39.861607+0000 mon.a (mon.0) 2435 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[17385]: cluster 2024-02-10T08:36:40.170260+0000 mgr.y (mgr.14547) 556 : cluster [DBG] pgmap v819: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:36:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[17385]: audit 2024-02-10T08:36:40.859884+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]': finished 2024-02-10T08:36:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[17385]: cluster 2024-02-10T08:36:40.860020+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[17385]: audit 2024-02-10T08:36:40.861637+0000 mon.c (mon.2) 843 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[17385]: audit 2024-02-10T08:36:40.864313+0000 mon.a (mon.0) 2438 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[24488]: cluster 2024-02-10T08:36:40.170260+0000 mgr.y (mgr.14547) 556 : cluster [DBG] pgmap v819: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[24488]: audit 2024-02-10T08:36:40.859884+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]': finished 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[24488]: cluster 2024-02-10T08:36:40.860020+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[24488]: audit 2024-02-10T08:36:40.861637+0000 mon.c (mon.2) 843 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:36:42.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:41 smithi067 bash[24488]: audit 2024-02-10T08:36:40.864313+0000 mon.a (mon.0) 2438 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:36:42.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:41 smithi134 bash[18722]: cluster 2024-02-10T08:36:40.170260+0000 mgr.y (mgr.14547) 556 : cluster [DBG] pgmap v819: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-10T08:36:42.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:41 smithi134 bash[18722]: audit 2024-02-10T08:36:40.859884+0000 mon.a (mon.0) 2436 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-88"}]': finished 2024-02-10T08:36:42.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:41 smithi134 bash[18722]: cluster 2024-02-10T08:36:40.860020+0000 mon.a (mon.0) 2437 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-10T08:36:42.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:41 smithi134 bash[18722]: audit 2024-02-10T08:36:40.861637+0000 mon.c (mon.2) 843 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:36:42.256 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:41 smithi134 bash[18722]: audit 2024-02-10T08:36:40.864313+0000 mon.a (mon.0) 2438 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:36:42.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:36:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: cluster 2024-02-10T08:36:41.857783+0000 mon.a (mon.0) 2439 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:36:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: audit 2024-02-10T08:36:41.868780+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: cluster 2024-02-10T08:36:41.869006+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: audit 2024-02-10T08:36:41.870543+0000 mon.c (mon.2) 844 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: audit 2024-02-10T08:36:41.872254+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: cluster 2024-02-10T08:36:42.171170+0000 mgr.y (mgr.14547) 557 : cluster [DBG] pgmap v822: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[17385]: audit 2024-02-10T08:36:42.255103+0000 mgr.y (mgr.14547) 558 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: cluster 2024-02-10T08:36:41.857783+0000 mon.a (mon.0) 2439 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: audit 2024-02-10T08:36:41.868780+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: cluster 2024-02-10T08:36:41.869006+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: audit 2024-02-10T08:36:41.870543+0000 mon.c (mon.2) 844 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: audit 2024-02-10T08:36:41.872254+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T08:36:43.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: cluster 2024-02-10T08:36:42.171170+0000 mgr.y (mgr.14547) 557 : cluster [DBG] pgmap v822: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:36:43.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:42 smithi067 bash[24488]: audit 2024-02-10T08:36:42.255103+0000 mgr.y (mgr.14547) 558 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:43.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: cluster 2024-02-10T08:36:41.857783+0000 mon.a (mon.0) 2439 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:36:43.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: audit 2024-02-10T08:36:41.868780+0000 mon.a (mon.0) 2440 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:36:43.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: cluster 2024-02-10T08:36:41.869006+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-10T08:36:43.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: audit 2024-02-10T08:36:41.870543+0000 mon.c (mon.2) 844 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T08:36:43.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: audit 2024-02-10T08:36:41.872254+0000 mon.a (mon.0) 2442 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-10T08:36:43.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: cluster 2024-02-10T08:36:42.171170+0000 mgr.y (mgr.14547) 557 : cluster [DBG] pgmap v822: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:36:43.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:42 smithi134 bash[18722]: audit 2024-02-10T08:36:42.255103+0000 mgr.y (mgr.14547) 558 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:36:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:36:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:43 smithi134 bash[18722]: audit 2024-02-10T08:36:42.869548+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]': finished 2024-02-10T08:36:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:43 smithi134 bash[18722]: cluster 2024-02-10T08:36:42.869667+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-10T08:36:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:43 smithi134 bash[18722]: audit 2024-02-10T08:36:42.873592+0000 mon.c (mon.2) 845 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:44.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:43 smithi134 bash[18722]: audit 2024-02-10T08:36:42.874250+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[17385]: audit 2024-02-10T08:36:42.869548+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]': finished 2024-02-10T08:36:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[17385]: cluster 2024-02-10T08:36:42.869667+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-10T08:36:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[17385]: audit 2024-02-10T08:36:42.873592+0000 mon.c (mon.2) 845 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[17385]: audit 2024-02-10T08:36:42.874250+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[24488]: audit 2024-02-10T08:36:42.869548+0000 mon.a (mon.0) 2443 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_count","val": "1"}]': finished 2024-02-10T08:36:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[24488]: cluster 2024-02-10T08:36:42.869667+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-10T08:36:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[24488]: audit 2024-02-10T08:36:42.873592+0000 mon.c (mon.2) 845 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:44.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:43 smithi067 bash[24488]: audit 2024-02-10T08:36:42.874250+0000 mon.a (mon.0) 2445 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:45.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:44 smithi134 bash[18722]: audit 2024-02-10T08:36:43.875698+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:36:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:44 smithi134 bash[18722]: cluster 2024-02-10T08:36:43.875882+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-10T08:36:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:44 smithi134 bash[18722]: audit 2024-02-10T08:36:43.880751+0000 mon.c (mon.2) 846 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T08:36:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:44 smithi134 bash[18722]: audit 2024-02-10T08:36:43.885923+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T08:36:45.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:44 smithi134 bash[18722]: cluster 2024-02-10T08:36:44.171976+0000 mgr.y (mgr.14547) 559 : cluster [DBG] pgmap v825: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:36:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[17385]: audit 2024-02-10T08:36:43.875698+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:36:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[17385]: cluster 2024-02-10T08:36:43.875882+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[17385]: audit 2024-02-10T08:36:43.880751+0000 mon.c (mon.2) 846 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[17385]: audit 2024-02-10T08:36:43.885923+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[17385]: cluster 2024-02-10T08:36:44.171976+0000 mgr.y (mgr.14547) 559 : cluster [DBG] pgmap v825: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[24488]: audit 2024-02-10T08:36:43.875698+0000 mon.a (mon.0) 2446 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[24488]: cluster 2024-02-10T08:36:43.875882+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[24488]: audit 2024-02-10T08:36:43.880751+0000 mon.c (mon.2) 846 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[24488]: audit 2024-02-10T08:36:43.885923+0000 mon.a (mon.0) 2448 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-10T08:36:45.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:44 smithi067 bash[24488]: cluster 2024-02-10T08:36:44.171976+0000 mgr.y (mgr.14547) 559 : cluster [DBG] pgmap v825: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:36:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:45 smithi134 bash[18722]: audit 2024-02-10T08:36:44.891424+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]': finished 2024-02-10T08:36:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:45 smithi134 bash[18722]: cluster 2024-02-10T08:36:44.891550+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-10T08:36:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:45 smithi134 bash[18722]: audit 2024-02-10T08:36:44.931672+0000 mon.c (mon.2) 847 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:46.278 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:45 smithi134 bash[18722]: audit 2024-02-10T08:36:44.932373+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[17385]: audit 2024-02-10T08:36:44.891424+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]': finished 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[17385]: cluster 2024-02-10T08:36:44.891550+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[17385]: audit 2024-02-10T08:36:44.931672+0000 mon.c (mon.2) 847 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[17385]: audit 2024-02-10T08:36:44.932373+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[24488]: audit 2024-02-10T08:36:44.891424+0000 mon.a (mon.0) 2449 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-88","var": "target_max_objects","val": "250"}]': finished 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[24488]: cluster 2024-02-10T08:36:44.891550+0000 mon.a (mon.0) 2450 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[24488]: audit 2024-02-10T08:36:44.931672+0000 mon.c (mon.2) 847 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:46.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:45 smithi067 bash[24488]: audit 2024-02-10T08:36:44.932373+0000 mon.a (mon.0) 2451 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:46 smithi134 bash[18722]: audit 2024-02-10T08:36:45.903320+0000 mon.a (mon.0) 2452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:46 smithi134 bash[18722]: cluster 2024-02-10T08:36:45.904225+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-10T08:36:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:46 smithi134 bash[18722]: audit 2024-02-10T08:36:45.905285+0000 mon.c (mon.2) 848 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:46 smithi134 bash[18722]: audit 2024-02-10T08:36:45.905850+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:46 smithi134 bash[18722]: cluster 2024-02-10T08:36:46.173046+0000 mgr.y (mgr.14547) 560 : cluster [DBG] pgmap v828: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:47.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:46 smithi134 bash[18722]: audit 2024-02-10T08:36:46.905338+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]': finished 2024-02-10T08:36:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[17385]: audit 2024-02-10T08:36:45.903320+0000 mon.a (mon.0) 2452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[17385]: cluster 2024-02-10T08:36:45.904225+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-10T08:36:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[17385]: audit 2024-02-10T08:36:45.905285+0000 mon.c (mon.2) 848 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[17385]: audit 2024-02-10T08:36:45.905850+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[17385]: cluster 2024-02-10T08:36:46.173046+0000 mgr.y (mgr.14547) 560 : cluster [DBG] pgmap v828: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[17385]: audit 2024-02-10T08:36:46.905338+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]': finished 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[24488]: audit 2024-02-10T08:36:45.903320+0000 mon.a (mon.0) 2452 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[24488]: cluster 2024-02-10T08:36:45.904225+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[24488]: audit 2024-02-10T08:36:45.905285+0000 mon.c (mon.2) 848 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[24488]: audit 2024-02-10T08:36:45.905850+0000 mon.a (mon.0) 2454 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]: dispatch 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[24488]: cluster 2024-02-10T08:36:46.173046+0000 mgr.y (mgr.14547) 560 : cluster [DBG] pgmap v828: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:46 smithi067 bash[24488]: audit 2024-02-10T08:36:46.905338+0000 mon.a (mon.0) 2455 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-88"}]': finished 2024-02-10T08:36:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:36:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:47] "GET /metrics HTTP/1.1" 200 189404 "" "Prometheus/2.33.4" 2024-02-10T08:36:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:47 smithi067 bash[24488]: cluster 2024-02-10T08:36:46.905473+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-10T08:36:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:47 smithi067 bash[17385]: cluster 2024-02-10T08:36:46.905473+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-10T08:36:48.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:47 smithi134 bash[18722]: cluster 2024-02-10T08:36:46.905473+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-10T08:36:49.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:48 smithi134 bash[18722]: cluster 2024-02-10T08:36:47.922891+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-10T08:36:49.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:48 smithi134 bash[18722]: cluster 2024-02-10T08:36:48.174056+0000 mgr.y (mgr.14547) 561 : cluster [DBG] pgmap v831: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:49.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:48 smithi067 bash[24488]: cluster 2024-02-10T08:36:47.922891+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-10T08:36:49.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:48 smithi067 bash[24488]: cluster 2024-02-10T08:36:48.174056+0000 mgr.y (mgr.14547) 561 : cluster [DBG] pgmap v831: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:48 smithi067 bash[17385]: cluster 2024-02-10T08:36:47.922891+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-10T08:36:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:48 smithi067 bash[17385]: cluster 2024-02-10T08:36:48.174056+0000 mgr.y (mgr.14547) 561 : cluster [DBG] pgmap v831: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:50.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:49 smithi134 bash[18722]: cluster 2024-02-10T08:36:48.937425+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-10T08:36:50.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:49 smithi134 bash[18722]: audit 2024-02-10T08:36:48.946786+0000 mon.c (mon.2) 849 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:50.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:49 smithi134 bash[18722]: audit 2024-02-10T08:36:48.949147+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:49 smithi067 bash[24488]: cluster 2024-02-10T08:36:48.937425+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-10T08:36:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:49 smithi067 bash[24488]: audit 2024-02-10T08:36:48.946786+0000 mon.c (mon.2) 849 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:49 smithi067 bash[24488]: audit 2024-02-10T08:36:48.949147+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:50.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:49 smithi067 bash[17385]: cluster 2024-02-10T08:36:48.937425+0000 mon.a (mon.0) 2458 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-10T08:36:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:49 smithi067 bash[17385]: audit 2024-02-10T08:36:48.946786+0000 mon.c (mon.2) 849 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:49 smithi067 bash[17385]: audit 2024-02-10T08:36:48.949147+0000 mon.a (mon.0) 2459 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:36:51.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:50 smithi134 bash[18722]: audit 2024-02-10T08:36:49.944137+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:51.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:50 smithi134 bash[18722]: cluster 2024-02-10T08:36:49.944304+0000 mon.a (mon.0) 2461 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-10T08:36:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:50 smithi134 bash[18722]: audit 2024-02-10T08:36:49.948219+0000 mon.c (mon.2) 850 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:50 smithi134 bash[18722]: audit 2024-02-10T08:36:49.958594+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:51.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:50 smithi134 bash[18722]: cluster 2024-02-10T08:36:50.174963+0000 mgr.y (mgr.14547) 562 : cluster [DBG] pgmap v834: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[24488]: audit 2024-02-10T08:36:49.944137+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[24488]: cluster 2024-02-10T08:36:49.944304+0000 mon.a (mon.0) 2461 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-10T08:36:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[24488]: audit 2024-02-10T08:36:49.948219+0000 mon.c (mon.2) 850 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[24488]: audit 2024-02-10T08:36:49.958594+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[24488]: cluster 2024-02-10T08:36:50.174963+0000 mgr.y (mgr.14547) 562 : cluster [DBG] pgmap v834: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[17385]: audit 2024-02-10T08:36:49.944137+0000 mon.a (mon.0) 2460 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:36:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[17385]: cluster 2024-02-10T08:36:49.944304+0000 mon.a (mon.0) 2461 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-10T08:36:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[17385]: audit 2024-02-10T08:36:49.948219+0000 mon.c (mon.2) 850 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[17385]: audit 2024-02-10T08:36:49.958594+0000 mon.a (mon.0) 2462 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:36:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:50 smithi067 bash[17385]: cluster 2024-02-10T08:36:50.174963+0000 mgr.y (mgr.14547) 562 : cluster [DBG] pgmap v834: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:52.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:51 smithi134 bash[18722]: audit 2024-02-10T08:36:50.958902+0000 mon.a (mon.0) 2463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:36:52.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:51 smithi134 bash[18722]: cluster 2024-02-10T08:36:50.959414+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-10T08:36:52.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:51 smithi134 bash[18722]: audit 2024-02-10T08:36:50.961547+0000 mon.c (mon.2) 851 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:52.264 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:51 smithi134 bash[18722]: audit 2024-02-10T08:36:50.968371+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[17385]: audit 2024-02-10T08:36:50.958902+0000 mon.a (mon.0) 2463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:36:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[17385]: cluster 2024-02-10T08:36:50.959414+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-10T08:36:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[17385]: audit 2024-02-10T08:36:50.961547+0000 mon.c (mon.2) 851 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[17385]: audit 2024-02-10T08:36:50.968371+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[24488]: audit 2024-02-10T08:36:50.958902+0000 mon.a (mon.0) 2463 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:36:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[24488]: cluster 2024-02-10T08:36:50.959414+0000 mon.a (mon.0) 2464 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-10T08:36:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[24488]: audit 2024-02-10T08:36:50.961547+0000 mon.c (mon.2) 851 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:51 smithi067 bash[24488]: audit 2024-02-10T08:36:50.968371+0000 mon.a (mon.0) 2465 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:52.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:36:53.276 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:36:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:36:53.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[18722]: audit 2024-02-10T08:36:51.972076+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]': finished 2024-02-10T08:36:53.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[18722]: cluster 2024-02-10T08:36:51.972236+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-10T08:36:53.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[18722]: audit 2024-02-10T08:36:51.974128+0000 mon.c (mon.2) 852 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]: dispatch 2024-02-10T08:36:53.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[18722]: audit 2024-02-10T08:36:51.979769+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]: dispatch 2024-02-10T08:36:53.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[18722]: cluster 2024-02-10T08:36:52.175736+0000 mgr.y (mgr.14547) 563 : cluster [DBG] pgmap v837: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:53.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:52 smithi134 bash[18722]: audit 2024-02-10T08:36:52.262915+0000 mgr.y (mgr.14547) 564 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[17385]: audit 2024-02-10T08:36:51.972076+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]': finished 2024-02-10T08:36:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[17385]: cluster 2024-02-10T08:36:51.972236+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-10T08:36:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[17385]: audit 2024-02-10T08:36:51.974128+0000 mon.c (mon.2) 852 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]: dispatch 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[17385]: audit 2024-02-10T08:36:51.979769+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]: dispatch 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[17385]: cluster 2024-02-10T08:36:52.175736+0000 mgr.y (mgr.14547) 563 : cluster [DBG] pgmap v837: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[17385]: audit 2024-02-10T08:36:52.262915+0000 mgr.y (mgr.14547) 564 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[24488]: audit 2024-02-10T08:36:51.972076+0000 mon.a (mon.0) 2466 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-6", "overlaypool": "test-rados-api-smithi067-54339-90"}]': finished 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[24488]: cluster 2024-02-10T08:36:51.972236+0000 mon.a (mon.0) 2467 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[24488]: audit 2024-02-10T08:36:51.974128+0000 mon.c (mon.2) 852 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]: dispatch 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[24488]: audit 2024-02-10T08:36:51.979769+0000 mon.a (mon.0) 2468 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]: dispatch 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[24488]: cluster 2024-02-10T08:36:52.175736+0000 mgr.y (mgr.14547) 563 : cluster [DBG] pgmap v837: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:52 smithi067 bash[24488]: audit 2024-02-10T08:36:52.262915+0000 mgr.y (mgr.14547) 564 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:36:54.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:54 smithi134 bash[18722]: cluster 2024-02-10T08:36:52.975168+0000 mon.a (mon.0) 2469 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:36:54.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:54 smithi134 bash[18722]: audit 2024-02-10T08:36:52.984695+0000 mon.a (mon.0) 2470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]': finished 2024-02-10T08:36:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:54 smithi134 bash[18722]: cluster 2024-02-10T08:36:52.984798+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-10T08:36:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:54 smithi134 bash[18722]: audit 2024-02-10T08:36:52.987667+0000 mon.c (mon.2) 853 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:36:54.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:54 smithi134 bash[18722]: audit 2024-02-10T08:36:52.994903+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:36:54.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[24488]: cluster 2024-02-10T08:36:52.975168+0000 mon.a (mon.0) 2469 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:36:54.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[24488]: audit 2024-02-10T08:36:52.984695+0000 mon.a (mon.0) 2470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]': finished 2024-02-10T08:36:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[24488]: cluster 2024-02-10T08:36:52.984798+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-10T08:36:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[24488]: audit 2024-02-10T08:36:52.987667+0000 mon.c (mon.2) 853 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:36:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[24488]: audit 2024-02-10T08:36:52.994903+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:36:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[17385]: cluster 2024-02-10T08:36:52.975168+0000 mon.a (mon.0) 2469 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:36:54.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[17385]: audit 2024-02-10T08:36:52.984695+0000 mon.a (mon.0) 2470 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-90", "mode": "writeback"}]': finished 2024-02-10T08:36:54.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[17385]: cluster 2024-02-10T08:36:52.984798+0000 mon.a (mon.0) 2471 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-10T08:36:54.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[17385]: audit 2024-02-10T08:36:52.987667+0000 mon.c (mon.2) 853 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:36:54.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:54 smithi067 bash[17385]: audit 2024-02-10T08:36:52.994903+0000 mon.a (mon.0) 2472 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:36:55.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:53.996653+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: cluster 2024-02-10T08:36:53.996843+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:53.999234+0000 mon.c (mon.2) 854 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:54.005786+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:54.041397+0000 mon.c (mon.2) 855 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: cluster 2024-02-10T08:36:54.176494+0000 mgr.y (mgr.14547) 565 : cluster [DBG] pgmap v840: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:54.320237+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:54.587939+0000 mon.c (mon.2) 856 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:54.590146+0000 mon.c (mon.2) 857 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:36:55.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:55 smithi134 bash[18722]: audit 2024-02-10T08:36:54.600213+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:36:55.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:53.996653+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: cluster 2024-02-10T08:36:53.996843+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:53.999234+0000 mon.c (mon.2) 854 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:54.005786+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:54.041397+0000 mon.c (mon.2) 855 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: cluster 2024-02-10T08:36:54.176494+0000 mgr.y (mgr.14547) 565 : cluster [DBG] pgmap v840: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:54.320237+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:54.587939+0000 mon.c (mon.2) 856 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:54.590146+0000 mon.c (mon.2) 857 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[17385]: audit 2024-02-10T08:36:54.600213+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:36:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:53.996653+0000 mon.a (mon.0) 2473 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: cluster 2024-02-10T08:36:53.996843+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:53.999234+0000 mon.c (mon.2) 854 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:54.005786+0000 mon.a (mon.0) 2475 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:54.041397+0000 mon.c (mon.2) 855 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: cluster 2024-02-10T08:36:54.176494+0000 mgr.y (mgr.14547) 565 : cluster [DBG] pgmap v840: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:54.320237+0000 mon.a (mon.0) 2476 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:54.587939+0000 mon.c (mon.2) 856 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:54.590146+0000 mon.c (mon.2) 857 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:36:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:55 smithi067 bash[24488]: audit 2024-02-10T08:36:54.600213+0000 mon.a (mon.0) 2477 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:36:56.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:56 smithi134 bash[18722]: audit 2024-02-10T08:36:55.007884+0000 mon.a (mon.0) 2478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:36:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:56 smithi134 bash[18722]: cluster 2024-02-10T08:36:55.008059+0000 mon.a (mon.0) 2479 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-10T08:36:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:56 smithi134 bash[18722]: audit 2024-02-10T08:36:55.010523+0000 mon.c (mon.2) 858 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:36:56.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:56 smithi134 bash[18722]: audit 2024-02-10T08:36:55.018382+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:36:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[24488]: audit 2024-02-10T08:36:55.007884+0000 mon.a (mon.0) 2478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:36:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[24488]: cluster 2024-02-10T08:36:55.008059+0000 mon.a (mon.0) 2479 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-10T08:36:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[24488]: audit 2024-02-10T08:36:55.010523+0000 mon.c (mon.2) 858 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:36:56.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[24488]: audit 2024-02-10T08:36:55.018382+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:36:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[17385]: audit 2024-02-10T08:36:55.007884+0000 mon.a (mon.0) 2478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:36:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[17385]: cluster 2024-02-10T08:36:55.008059+0000 mon.a (mon.0) 2479 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-10T08:36:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[17385]: audit 2024-02-10T08:36:55.010523+0000 mon.c (mon.2) 858 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:36:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:56 smithi067 bash[17385]: audit 2024-02-10T08:36:55.018382+0000 mon.a (mon.0) 2480 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:36:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17385]: cluster 2024-02-10T08:36:56.011613+0000 mon.a (mon.0) 2481 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:36:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17385]: audit 2024-02-10T08:36:56.020342+0000 mon.a (mon.0) 2482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:36:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17385]: cluster 2024-02-10T08:36:56.020446+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-10T08:36:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17385]: audit 2024-02-10T08:36:56.022581+0000 mon.c (mon.2) 859 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17385]: audit 2024-02-10T08:36:56.029183+0000 mon.a (mon.0) 2484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17385]: cluster 2024-02-10T08:36:56.177294+0000 mgr.y (mgr.14547) 566 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[24488]: cluster 2024-02-10T08:36:56.011613+0000 mon.a (mon.0) 2481 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[24488]: audit 2024-02-10T08:36:56.020342+0000 mon.a (mon.0) 2482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[24488]: cluster 2024-02-10T08:36:56.020446+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[24488]: audit 2024-02-10T08:36:56.022581+0000 mon.c (mon.2) 859 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[24488]: audit 2024-02-10T08:36:56.029183+0000 mon.a (mon.0) 2484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T08:36:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[24488]: cluster 2024-02-10T08:36:56.177294+0000 mgr.y (mgr.14547) 566 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:57 smithi134 bash[18722]: cluster 2024-02-10T08:36:56.011613+0000 mon.a (mon.0) 2481 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:36:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:57 smithi134 bash[18722]: audit 2024-02-10T08:36:56.020342+0000 mon.a (mon.0) 2482 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:36:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:57 smithi134 bash[18722]: cluster 2024-02-10T08:36:56.020446+0000 mon.a (mon.0) 2483 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-10T08:36:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:57 smithi134 bash[18722]: audit 2024-02-10T08:36:56.022581+0000 mon.c (mon.2) 859 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T08:36:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:57 smithi134 bash[18722]: audit 2024-02-10T08:36:56.029183+0000 mon.a (mon.0) 2484 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-10T08:36:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:57 smithi134 bash[18722]: cluster 2024-02-10T08:36:56.177294+0000 mgr.y (mgr.14547) 566 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:36:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:36:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:36:57] "GET /metrics HTTP/1.1" 200 189404 "" "Prometheus/2.33.4" 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[17385]: audit 2024-02-10T08:36:57.033003+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[17385]: cluster 2024-02-10T08:36:57.033171+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[17385]: audit 2024-02-10T08:36:57.080481+0000 mon.c (mon.2) 860 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[17385]: audit 2024-02-10T08:36:57.081151+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[24488]: audit 2024-02-10T08:36:57.033003+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[24488]: cluster 2024-02-10T08:36:57.033171+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[24488]: audit 2024-02-10T08:36:57.080481+0000 mon.c (mon.2) 860 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:58.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:58 smithi067 bash[24488]: audit 2024-02-10T08:36:57.081151+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:58 smithi134 bash[18722]: audit 2024-02-10T08:36:57.033003+0000 mon.a (mon.0) 2485 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-10T08:36:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:58 smithi134 bash[18722]: cluster 2024-02-10T08:36:57.033171+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-10T08:36:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:58 smithi134 bash[18722]: audit 2024-02-10T08:36:57.080481+0000 mon.c (mon.2) 860 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:58 smithi134 bash[18722]: audit 2024-02-10T08:36:57.081151+0000 mon.a (mon.0) 2487 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:36:59.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[17385]: audit 2024-02-10T08:36:58.052605+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[17385]: cluster 2024-02-10T08:36:58.052702+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-10T08:36:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[17385]: audit 2024-02-10T08:36:58.058926+0000 mon.c (mon.2) 861 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[17385]: audit 2024-02-10T08:36:58.059855+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:59.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[17385]: cluster 2024-02-10T08:36:58.178141+0000 mgr.y (mgr.14547) 567 : cluster [DBG] pgmap v846: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:59.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[24488]: audit 2024-02-10T08:36:58.052605+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[24488]: cluster 2024-02-10T08:36:58.052702+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-10T08:36:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[24488]: audit 2024-02-10T08:36:58.058926+0000 mon.c (mon.2) 861 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[24488]: audit 2024-02-10T08:36:58.059855+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:59.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:36:59 smithi067 bash[24488]: cluster 2024-02-10T08:36:58.178141+0000 mgr.y (mgr.14547) 567 : cluster [DBG] pgmap v846: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:36:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:59 smithi134 bash[18722]: audit 2024-02-10T08:36:58.052605+0000 mon.a (mon.0) 2488 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]': finished 2024-02-10T08:36:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:59 smithi134 bash[18722]: cluster 2024-02-10T08:36:58.052702+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-10T08:36:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:59 smithi134 bash[18722]: audit 2024-02-10T08:36:58.058926+0000 mon.c (mon.2) 861 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:59 smithi134 bash[18722]: audit 2024-02-10T08:36:58.059855+0000 mon.a (mon.0) 2490 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]: dispatch 2024-02-10T08:36:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:36:59 smithi134 bash[18722]: cluster 2024-02-10T08:36:58.178141+0000 mgr.y (mgr.14547) 567 : cluster [DBG] pgmap v846: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:00 smithi067 bash[24488]: audit 2024-02-10T08:36:59.056955+0000 mon.a (mon.0) 2491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]': finished 2024-02-10T08:37:00.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:00 smithi067 bash[24488]: cluster 2024-02-10T08:36:59.057093+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-10T08:37:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:00 smithi067 bash[17385]: audit 2024-02-10T08:36:59.056955+0000 mon.a (mon.0) 2491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]': finished 2024-02-10T08:37:00.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:00 smithi067 bash[17385]: cluster 2024-02-10T08:36:59.057093+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-10T08:37:00.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:00 smithi134 bash[18722]: audit 2024-02-10T08:36:59.056955+0000 mon.a (mon.0) 2491 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-90"}]': finished 2024-02-10T08:37:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:00 smithi134 bash[18722]: cluster 2024-02-10T08:36:59.057093+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-10T08:37:01.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:01 smithi067 bash[24488]: cluster 2024-02-10T08:37:00.075652+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-10T08:37:01.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:01 smithi067 bash[24488]: cluster 2024-02-10T08:37:00.178968+0000 mgr.y (mgr.14547) 568 : cluster [DBG] pgmap v849: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:01 smithi067 bash[17385]: cluster 2024-02-10T08:37:00.075652+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-10T08:37:01.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:01 smithi067 bash[17385]: cluster 2024-02-10T08:37:00.178968+0000 mgr.y (mgr.14547) 568 : cluster [DBG] pgmap v849: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:01 smithi134 bash[18722]: cluster 2024-02-10T08:37:00.075652+0000 mon.a (mon.0) 2493 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-10T08:37:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:01 smithi134 bash[18722]: cluster 2024-02-10T08:37:00.178968+0000 mgr.y (mgr.14547) 568 : cluster [DBG] pgmap v849: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[17385]: cluster 2024-02-10T08:37:01.082739+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-10T08:37:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[17385]: audit 2024-02-10T08:37:01.084796+0000 mon.c (mon.2) 862 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[17385]: audit 2024-02-10T08:37:01.087096+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:02.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[17385]: audit 2024-02-10T08:37:02.083753+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:02.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[17385]: cluster 2024-02-10T08:37:02.084394+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-10T08:37:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[24488]: cluster 2024-02-10T08:37:01.082739+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-10T08:37:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[24488]: audit 2024-02-10T08:37:01.084796+0000 mon.c (mon.2) 862 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[24488]: audit 2024-02-10T08:37:01.087096+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[24488]: audit 2024-02-10T08:37:02.083753+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:02.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:02 smithi067 bash[24488]: cluster 2024-02-10T08:37:02.084394+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-10T08:37:02.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:37:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:37:02.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:02 smithi134 bash[18722]: cluster 2024-02-10T08:37:01.082739+0000 mon.a (mon.0) 2494 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-10T08:37:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:02 smithi134 bash[18722]: audit 2024-02-10T08:37:01.084796+0000 mon.c (mon.2) 862 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:02 smithi134 bash[18722]: audit 2024-02-10T08:37:01.087096+0000 mon.a (mon.0) 2495 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:02 smithi134 bash[18722]: audit 2024-02-10T08:37:02.083753+0000 mon.a (mon.0) 2496 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:02 smithi134 bash[18722]: cluster 2024-02-10T08:37:02.084394+0000 mon.a (mon.0) 2497 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-10T08:37:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:02.087284+0000 mon.c (mon.2) 863 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:37:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:02.092355+0000 mon.c (mon.2) 864 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:37:03.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:02.093139+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: cluster 2024-02-10T08:37:02.179753+0000 mgr.y (mgr.14547) 569 : cluster [DBG] pgmap v852: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:02.274406+0000 mgr.y (mgr.14547) 570 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:03.085767+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: cluster 2024-02-10T08:37:03.085985+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:03.088620+0000 mon.c (mon.2) 865 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[17385]: audit 2024-02-10T08:37:03.089499+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:02.087284+0000 mon.c (mon.2) 863 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:02.092355+0000 mon.c (mon.2) 864 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:02.093139+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:37:03.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: cluster 2024-02-10T08:37:02.179753+0000 mgr.y (mgr.14547) 569 : cluster [DBG] pgmap v852: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:02.274406+0000 mgr.y (mgr.14547) 570 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:03.085767+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:37:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: cluster 2024-02-10T08:37:03.085985+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-10T08:37:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:03.088620+0000 mon.c (mon.2) 865 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:37:03.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:03 smithi067 bash[24488]: audit 2024-02-10T08:37:03.089499+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:37:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:37:03.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:02.087284+0000 mon.c (mon.2) 863 : audit [DBG] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:02.092355+0000 mon.c (mon.2) 864 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:02.093139+0000 mon.a (mon.0) 2498 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: cluster 2024-02-10T08:37:02.179753+0000 mgr.y (mgr.14547) 569 : cluster [DBG] pgmap v852: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:02.274406+0000 mgr.y (mgr.14547) 570 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:03.085767+0000 mon.a (mon.0) 2499 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: cluster 2024-02-10T08:37:03.085985+0000 mon.a (mon.0) 2500 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:03.088620+0000 mon.c (mon.2) 865 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:37:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:03 smithi134 bash[18722]: audit 2024-02-10T08:37:03.089499+0000 mon.a (mon.0) 2501 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-10T08:37:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[24488]: audit 2024-02-10T08:37:04.086009+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:37:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[24488]: cluster 2024-02-10T08:37:04.086098+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-10T08:37:05.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[24488]: audit 2024-02-10T08:37:04.093746+0000 mon.c (mon.2) 866 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[24488]: audit 2024-02-10T08:37:04.094181+0000 mon.a (mon.0) 2504 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[24488]: cluster 2024-02-10T08:37:04.180732+0000 mgr.y (mgr.14547) 571 : cluster [DBG] pgmap v855: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[17385]: audit 2024-02-10T08:37:04.086009+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[17385]: cluster 2024-02-10T08:37:04.086098+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[17385]: audit 2024-02-10T08:37:04.093746+0000 mon.c (mon.2) 866 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[17385]: audit 2024-02-10T08:37:04.094181+0000 mon.a (mon.0) 2504 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:37:05.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:05 smithi067 bash[17385]: cluster 2024-02-10T08:37:04.180732+0000 mgr.y (mgr.14547) 571 : cluster [DBG] pgmap v855: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:05.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:05 smithi134 bash[18722]: audit 2024-02-10T08:37:04.086009+0000 mon.a (mon.0) 2502 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-10T08:37:05.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:05 smithi134 bash[18722]: cluster 2024-02-10T08:37:04.086098+0000 mon.a (mon.0) 2503 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-10T08:37:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:05 smithi134 bash[18722]: audit 2024-02-10T08:37:04.093746+0000 mon.c (mon.2) 866 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:37:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:05 smithi134 bash[18722]: audit 2024-02-10T08:37:04.094181+0000 mon.a (mon.0) 2504 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-10T08:37:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:05 smithi134 bash[18722]: cluster 2024-02-10T08:37:04.180732+0000 mgr.y (mgr.14547) 571 : cluster [DBG] pgmap v855: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[24488]: audit 2024-02-10T08:37:05.099029+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:37:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[24488]: cluster 2024-02-10T08:37:05.099149+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-10T08:37:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[24488]: audit 2024-02-10T08:37:05.150837+0000 mon.c (mon.2) 867 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[24488]: audit 2024-02-10T08:37:05.151465+0000 mon.a (mon.0) 2507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[24488]: audit 2024-02-10T08:37:05.152479+0000 mon.c (mon.2) 868 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-92"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[24488]: audit 2024-02-10T08:37:05.152962+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-92"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[17385]: audit 2024-02-10T08:37:05.099029+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[17385]: cluster 2024-02-10T08:37:05.099149+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[17385]: audit 2024-02-10T08:37:05.150837+0000 mon.c (mon.2) 867 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[17385]: audit 2024-02-10T08:37:05.151465+0000 mon.a (mon.0) 2507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[17385]: audit 2024-02-10T08:37:05.152479+0000 mon.c (mon.2) 868 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-92"}]: dispatch 2024-02-10T08:37:06.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:06 smithi067 bash[17385]: audit 2024-02-10T08:37:05.152962+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-92"}]: dispatch 2024-02-10T08:37:06.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:06 smithi134 bash[18722]: audit 2024-02-10T08:37:05.099029+0000 mon.a (mon.0) 2505 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-10T08:37:06.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:06 smithi134 bash[18722]: cluster 2024-02-10T08:37:05.099149+0000 mon.a (mon.0) 2506 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-10T08:37:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:06 smithi134 bash[18722]: audit 2024-02-10T08:37:05.150837+0000 mon.c (mon.2) 867 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:37:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:06 smithi134 bash[18722]: audit 2024-02-10T08:37:05.151465+0000 mon.a (mon.0) 2507 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-6"}]: dispatch 2024-02-10T08:37:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:06 smithi134 bash[18722]: audit 2024-02-10T08:37:05.152479+0000 mon.c (mon.2) 868 : audit [INF] from='client.? 172.21.15.67:0/1362234834' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-92"}]: dispatch 2024-02-10T08:37:06.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:06 smithi134 bash[18722]: audit 2024-02-10T08:37:05.152962+0000 mon.a (mon.0) 2508 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-6", "tierpool": "test-rados-api-smithi067-54339-92"}]: dispatch 2024-02-10T08:37:07.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:07 smithi067 bash[24488]: cluster 2024-02-10T08:37:06.109722+0000 mon.a (mon.0) 2509 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-10T08:37:07.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:07 smithi067 bash[24488]: cluster 2024-02-10T08:37:06.181635+0000 mgr.y (mgr.14547) 572 : cluster [DBG] pgmap v858: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:07.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:07 smithi067 bash[17385]: cluster 2024-02-10T08:37:06.109722+0000 mon.a (mon.0) 2509 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-10T08:37:07.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:07 smithi067 bash[17385]: cluster 2024-02-10T08:37:06.181635+0000 mgr.y (mgr.14547) 572 : cluster [DBG] pgmap v858: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:07.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:07 smithi134 bash[18722]: cluster 2024-02-10T08:37:06.109722+0000 mon.a (mon.0) 2509 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-10T08:37:07.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:07 smithi134 bash[18722]: cluster 2024-02-10T08:37:06.181635+0000 mgr.y (mgr.14547) 572 : cluster [DBG] pgmap v858: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:37:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:07] "GET /metrics HTTP/1.1" 200 186980 "" "Prometheus/2.33.4" 2024-02-10T08:37:08.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:08 smithi134 bash[18722]: cluster 2024-02-10T08:37:07.121395+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-10T08:37:08.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:08 smithi134 bash[18722]: audit 2024-02-10T08:37:07.145526+0000 mon.a (mon.0) 2511 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:08.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:08 smithi134 bash[18722]: audit 2024-02-10T08:37:07.146937+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:08.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:08 smithi134 bash[18722]: audit 2024-02-10T08:37:07.147630+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:08.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[17385]: cluster 2024-02-10T08:37:07.121395+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-10T08:37:08.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[17385]: audit 2024-02-10T08:37:07.145526+0000 mon.a (mon.0) 2511 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:08.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[17385]: audit 2024-02-10T08:37:07.146937+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:08.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[17385]: audit 2024-02-10T08:37:07.147630+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:08.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[24488]: cluster 2024-02-10T08:37:07.121395+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-10T08:37:08.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[24488]: audit 2024-02-10T08:37:07.145526+0000 mon.a (mon.0) 2511 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:08.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[24488]: audit 2024-02-10T08:37:07.146937+0000 mon.a (mon.0) 2512 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:08.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:08 smithi067 bash[24488]: audit 2024-02-10T08:37:07.147630+0000 mon.a (mon.0) 2513 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[17385]: audit 2024-02-10T08:37:08.124778+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[17385]: cluster 2024-02-10T08:37:08.124914+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-10T08:37:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[17385]: audit 2024-02-10T08:37:08.125938+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi067-54339-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[17385]: cluster 2024-02-10T08:37:08.182388+0000 mgr.y (mgr.14547) 573 : cluster [DBG] pgmap v861: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[24488]: audit 2024-02-10T08:37:08.124778+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[24488]: cluster 2024-02-10T08:37:08.124914+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-10T08:37:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[24488]: audit 2024-02-10T08:37:08.125938+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi067-54339-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:09 smithi067 bash[24488]: cluster 2024-02-10T08:37:08.182388+0000 mgr.y (mgr.14547) 573 : cluster [DBG] pgmap v861: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:09 smithi134 bash[18722]: audit 2024-02-10T08:37:08.124778+0000 mon.a (mon.0) 2514 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:09 smithi134 bash[18722]: cluster 2024-02-10T08:37:08.124914+0000 mon.a (mon.0) 2515 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-10T08:37:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:09 smithi134 bash[18722]: audit 2024-02-10T08:37:08.125938+0000 mon.a (mon.0) 2516 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi067-54339-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:10.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:09 smithi134 bash[18722]: cluster 2024-02-10T08:37:08.182388+0000 mgr.y (mgr.14547) 573 : cluster [DBG] pgmap v861: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:10 smithi067 bash[17385]: cluster 2024-02-10T08:37:09.575218+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-10T08:37:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:10 smithi067 bash[17385]: audit 2024-02-10T08:37:10.131662+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi067-54339-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:10.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:10 smithi067 bash[17385]: cluster 2024-02-10T08:37:10.131851+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-10T08:37:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:10 smithi067 bash[24488]: cluster 2024-02-10T08:37:09.575218+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-10T08:37:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:10 smithi067 bash[24488]: audit 2024-02-10T08:37:10.131662+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi067-54339-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:10 smithi067 bash[24488]: cluster 2024-02-10T08:37:10.131851+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-10T08:37:11.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:10 smithi134 bash[18722]: cluster 2024-02-10T08:37:09.575218+0000 mon.a (mon.0) 2517 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-10T08:37:11.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:10 smithi134 bash[18722]: audit 2024-02-10T08:37:10.131662+0000 mon.a (mon.0) 2518 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi067-54339-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:11.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:10 smithi134 bash[18722]: cluster 2024-02-10T08:37:10.131851+0000 mon.a (mon.0) 2519 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-10T08:37:12.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:11 smithi134 bash[18722]: cluster 2024-02-10T08:37:10.183204+0000 mgr.y (mgr.14547) 574 : cluster [DBG] pgmap v864: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:12.040 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:11 smithi134 bash[18722]: cluster 2024-02-10T08:37:11.136783+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-10T08:37:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:11 smithi067 bash[17385]: cluster 2024-02-10T08:37:10.183204+0000 mgr.y (mgr.14547) 574 : cluster [DBG] pgmap v864: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:11 smithi067 bash[17385]: cluster 2024-02-10T08:37:11.136783+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-10T08:37:12.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:11 smithi067 bash[24488]: cluster 2024-02-10T08:37:10.183204+0000 mgr.y (mgr.14547) 574 : cluster [DBG] pgmap v864: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:12.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:11 smithi067 bash[24488]: cluster 2024-02-10T08:37:11.136783+0000 mon.a (mon.0) 2520 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-10T08:37:12.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:37:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:37:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:37:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:37:13.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[17385]: cluster 2024-02-10T08:37:12.146318+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-10T08:37:13.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[17385]: audit 2024-02-10T08:37:12.148630+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:13.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[17385]: cluster 2024-02-10T08:37:12.183721+0000 mgr.y (mgr.14547) 575 : cluster [DBG] pgmap v867: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:13.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[17385]: audit 2024-02-10T08:37:12.284884+0000 mgr.y (mgr.14547) 576 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[24488]: cluster 2024-02-10T08:37:12.146318+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-10T08:37:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[24488]: audit 2024-02-10T08:37:12.148630+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[24488]: cluster 2024-02-10T08:37:12.183721+0000 mgr.y (mgr.14547) 575 : cluster [DBG] pgmap v867: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:13.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:13 smithi067 bash[24488]: audit 2024-02-10T08:37:12.284884+0000 mgr.y (mgr.14547) 576 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:14.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:13 smithi134 bash[18722]: cluster 2024-02-10T08:37:12.146318+0000 mon.a (mon.0) 2521 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-10T08:37:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:13 smithi134 bash[18722]: audit 2024-02-10T08:37:12.148630+0000 mon.a (mon.0) 2522 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:13 smithi134 bash[18722]: cluster 2024-02-10T08:37:12.183721+0000 mgr.y (mgr.14547) 575 : cluster [DBG] pgmap v867: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:14.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:13 smithi134 bash[18722]: audit 2024-02-10T08:37:12.284884+0000 mgr.y (mgr.14547) 576 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[24488]: cluster 2024-02-10T08:37:13.139543+0000 mon.a (mon.0) 2523 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[24488]: audit 2024-02-10T08:37:13.147966+0000 mon.a (mon.0) 2524 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[24488]: cluster 2024-02-10T08:37:13.148624+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-10T08:37:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[24488]: cluster 2024-02-10T08:37:14.151807+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-10T08:37:14.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[24488]: audit 2024-02-10T08:37:14.158292+0000 mon.a (mon.0) 2527 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[17385]: cluster 2024-02-10T08:37:13.139543+0000 mon.a (mon.0) 2523 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[17385]: audit 2024-02-10T08:37:13.147966+0000 mon.a (mon.0) 2524 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[17385]: cluster 2024-02-10T08:37:13.148624+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-10T08:37:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[17385]: cluster 2024-02-10T08:37:14.151807+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-10T08:37:14.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:14 smithi067 bash[17385]: audit 2024-02-10T08:37:14.158292+0000 mon.a (mon.0) 2527 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:14 smithi134 bash[18722]: cluster 2024-02-10T08:37:13.139543+0000 mon.a (mon.0) 2523 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:14 smithi134 bash[18722]: audit 2024-02-10T08:37:13.147966+0000 mon.a (mon.0) 2524 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:14 smithi134 bash[18722]: cluster 2024-02-10T08:37:13.148624+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-10T08:37:15.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:14 smithi134 bash[18722]: cluster 2024-02-10T08:37:14.151807+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-10T08:37:15.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:14 smithi134 bash[18722]: audit 2024-02-10T08:37:14.158292+0000 mon.a (mon.0) 2527 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[17385]: cluster 2024-02-10T08:37:14.184193+0000 mgr.y (mgr.14547) 577 : cluster [DBG] pgmap v870: 236 pgs: 64 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[17385]: audit 2024-02-10T08:37:15.154034+0000 mon.a (mon.0) 2528 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[17385]: cluster 2024-02-10T08:37:15.154339+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-10T08:37:15.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[17385]: audit 2024-02-10T08:37:15.158392+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[24488]: cluster 2024-02-10T08:37:14.184193+0000 mgr.y (mgr.14547) 577 : cluster [DBG] pgmap v870: 236 pgs: 64 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[24488]: audit 2024-02-10T08:37:15.154034+0000 mon.a (mon.0) 2528 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[24488]: cluster 2024-02-10T08:37:15.154339+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-10T08:37:15.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:15 smithi067 bash[24488]: audit 2024-02-10T08:37:15.158392+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:15 smithi134 bash[18722]: cluster 2024-02-10T08:37:14.184193+0000 mgr.y (mgr.14547) 577 : cluster [DBG] pgmap v870: 236 pgs: 64 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:37:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:15 smithi134 bash[18722]: audit 2024-02-10T08:37:15.154034+0000 mon.a (mon.0) 2528 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:15 smithi134 bash[18722]: cluster 2024-02-10T08:37:15.154339+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-10T08:37:16.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:15 smithi134 bash[18722]: audit 2024-02-10T08:37:15.158392+0000 mon.a (mon.0) 2530 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:17 smithi134 bash[18722]: audit 2024-02-10T08:37:16.158907+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:17 smithi134 bash[18722]: cluster 2024-02-10T08:37:16.159033+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-10T08:37:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:17 smithi134 bash[18722]: audit 2024-02-10T08:37:16.160239+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-97", "overlaypool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:17.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:17 smithi134 bash[18722]: cluster 2024-02-10T08:37:16.184937+0000 mgr.y (mgr.14547) 578 : cluster [DBG] pgmap v873: 236 pgs: 64 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[17385]: audit 2024-02-10T08:37:16.158907+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[17385]: cluster 2024-02-10T08:37:16.159033+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-10T08:37:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[17385]: audit 2024-02-10T08:37:16.160239+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-97", "overlaypool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:17.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[17385]: cluster 2024-02-10T08:37:16.184937+0000 mgr.y (mgr.14547) 578 : cluster [DBG] pgmap v873: 236 pgs: 64 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[24488]: audit 2024-02-10T08:37:16.158907+0000 mon.a (mon.0) 2531 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[24488]: cluster 2024-02-10T08:37:16.159033+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-10T08:37:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[24488]: audit 2024-02-10T08:37:16.160239+0000 mon.a (mon.0) 2533 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-97", "overlaypool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:17.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[24488]: cluster 2024-02-10T08:37:16.184937+0000 mgr.y (mgr.14547) 578 : cluster [DBG] pgmap v873: 236 pgs: 64 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:37:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:17] "GET /metrics HTTP/1.1" 200 192140 "" "Prometheus/2.33.4" 2024-02-10T08:37:18.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:18 smithi134 bash[18722]: audit 2024-02-10T08:37:17.163495+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-97", "overlaypool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:18.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:18 smithi134 bash[18722]: cluster 2024-02-10T08:37:17.163611+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-10T08:37:18.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:18 smithi134 bash[18722]: audit 2024-02-10T08:37:17.164720+0000 mon.a (mon.0) 2536 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-97-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:37:18.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:18 smithi067 bash[17385]: audit 2024-02-10T08:37:17.163495+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-97", "overlaypool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:18.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:18 smithi067 bash[17385]: cluster 2024-02-10T08:37:17.163611+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-10T08:37:18.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:18 smithi067 bash[17385]: audit 2024-02-10T08:37:17.164720+0000 mon.a (mon.0) 2536 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-97-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:37:18.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:18 smithi067 bash[24488]: audit 2024-02-10T08:37:17.163495+0000 mon.a (mon.0) 2534 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-97", "overlaypool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:18.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:18 smithi067 bash[24488]: cluster 2024-02-10T08:37:17.163611+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-10T08:37:18.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:18 smithi067 bash[24488]: audit 2024-02-10T08:37:17.164720+0000 mon.a (mon.0) 2536 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-97-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:37:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:19 smithi134 bash[18722]: cluster 2024-02-10T08:37:18.160483+0000 mon.a (mon.0) 2537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:19 smithi134 bash[18722]: audit 2024-02-10T08:37:18.167015+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-97-cache", "mode": "writeback"}]': finished 2024-02-10T08:37:19.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:19 smithi134 bash[18722]: cluster 2024-02-10T08:37:18.167128+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-10T08:37:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:19 smithi134 bash[18722]: cluster 2024-02-10T08:37:18.185444+0000 mgr.y (mgr.14547) 579 : cluster [DBG] pgmap v876: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:19.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:19 smithi134 bash[18722]: audit 2024-02-10T08:37:18.192321+0000 mon.a (mon.0) 2540 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-97"}]: dispatch 2024-02-10T08:37:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[17385]: cluster 2024-02-10T08:37:18.160483+0000 mon.a (mon.0) 2537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[17385]: audit 2024-02-10T08:37:18.167015+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-97-cache", "mode": "writeback"}]': finished 2024-02-10T08:37:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[17385]: cluster 2024-02-10T08:37:18.167128+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-10T08:37:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[17385]: cluster 2024-02-10T08:37:18.185444+0000 mgr.y (mgr.14547) 579 : cluster [DBG] pgmap v876: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:19.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[17385]: audit 2024-02-10T08:37:18.192321+0000 mon.a (mon.0) 2540 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-97"}]: dispatch 2024-02-10T08:37:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[24488]: cluster 2024-02-10T08:37:18.160483+0000 mon.a (mon.0) 2537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[24488]: audit 2024-02-10T08:37:18.167015+0000 mon.a (mon.0) 2538 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-97-cache", "mode": "writeback"}]': finished 2024-02-10T08:37:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[24488]: cluster 2024-02-10T08:37:18.167128+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-10T08:37:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[24488]: cluster 2024-02-10T08:37:18.185444+0000 mgr.y (mgr.14547) 579 : cluster [DBG] pgmap v876: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:19.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:19 smithi067 bash[24488]: audit 2024-02-10T08:37:18.192321+0000 mon.a (mon.0) 2540 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-97"}]: dispatch 2024-02-10T08:37:20.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:20 smithi134 bash[18722]: audit 2024-02-10T08:37:19.186193+0000 mon.a (mon.0) 2541 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-97"}]': finished 2024-02-10T08:37:20.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:20 smithi134 bash[18722]: cluster 2024-02-10T08:37:19.186302+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-10T08:37:20.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:20 smithi134 bash[18722]: audit 2024-02-10T08:37:19.188218+0000 mon.a (mon.0) 2543 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:20.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:20 smithi067 bash[24488]: audit 2024-02-10T08:37:19.186193+0000 mon.a (mon.0) 2541 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-97"}]': finished 2024-02-10T08:37:20.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:20 smithi067 bash[24488]: cluster 2024-02-10T08:37:19.186302+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-10T08:37:20.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:20 smithi067 bash[24488]: audit 2024-02-10T08:37:19.188218+0000 mon.a (mon.0) 2543 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:20 smithi067 bash[17385]: audit 2024-02-10T08:37:19.186193+0000 mon.a (mon.0) 2541 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-97"}]': finished 2024-02-10T08:37:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:20 smithi067 bash[17385]: cluster 2024-02-10T08:37:19.186302+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-10T08:37:20.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:20 smithi067 bash[17385]: audit 2024-02-10T08:37:19.188218+0000 mon.a (mon.0) 2543 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]: dispatch 2024-02-10T08:37:21.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:21 smithi134 bash[18722]: cluster 2024-02-10T08:37:20.181149+0000 mon.a (mon.0) 2544 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:37:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:21 smithi134 bash[18722]: cluster 2024-02-10T08:37:20.187030+0000 mgr.y (mgr.14547) 580 : cluster [DBG] pgmap v879: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:21 smithi134 bash[18722]: audit 2024-02-10T08:37:20.190412+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:21.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:21 smithi134 bash[18722]: cluster 2024-02-10T08:37:20.190546+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-10T08:37:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[17385]: cluster 2024-02-10T08:37:20.181149+0000 mon.a (mon.0) 2544 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:37:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[17385]: cluster 2024-02-10T08:37:20.187030+0000 mgr.y (mgr.14547) 580 : cluster [DBG] pgmap v879: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[17385]: audit 2024-02-10T08:37:20.190412+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:21.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[17385]: cluster 2024-02-10T08:37:20.190546+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-10T08:37:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[24488]: cluster 2024-02-10T08:37:20.181149+0000 mon.a (mon.0) 2544 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:37:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[24488]: cluster 2024-02-10T08:37:20.187030+0000 mgr.y (mgr.14547) 580 : cluster [DBG] pgmap v879: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:21.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[24488]: audit 2024-02-10T08:37:20.190412+0000 mon.a (mon.0) 2545 : audit [INF] from='client.? 172.21.15.67:0/4033602855' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-97", "tierpool": "test-rados-api-smithi067-54339-97-cache"}]': finished 2024-02-10T08:37:21.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:21 smithi067 bash[24488]: cluster 2024-02-10T08:37:20.190546+0000 mon.a (mon.0) 2546 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-10T08:37:22.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:22 smithi134 bash[18722]: cluster 2024-02-10T08:37:21.196176+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-10T08:37:22.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:37:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:37:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:22 smithi067 bash[17385]: cluster 2024-02-10T08:37:21.196176+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-10T08:37:22.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:22 smithi067 bash[24488]: cluster 2024-02-10T08:37:21.196176+0000 mon.a (mon.0) 2547 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-10T08:37:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:37:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[18722]: cluster 2024-02-10T08:37:22.189165+0000 mgr.y (mgr.14547) 581 : cluster [DBG] pgmap v881: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-02-10T08:37:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[18722]: cluster 2024-02-10T08:37:22.205339+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-10T08:37:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[18722]: audit 2024-02-10T08:37:22.238082+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[18722]: audit 2024-02-10T08:37:22.239219+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[18722]: audit 2024-02-10T08:37:22.239663+0000 mon.a (mon.0) 2551 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:23 smithi134 bash[18722]: audit 2024-02-10T08:37:22.296412+0000 mgr.y (mgr.14547) 582 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[17385]: cluster 2024-02-10T08:37:22.189165+0000 mgr.y (mgr.14547) 581 : cluster [DBG] pgmap v881: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-02-10T08:37:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[17385]: cluster 2024-02-10T08:37:22.205339+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-10T08:37:23.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[17385]: audit 2024-02-10T08:37:22.238082+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[17385]: audit 2024-02-10T08:37:22.239219+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[17385]: audit 2024-02-10T08:37:22.239663+0000 mon.a (mon.0) 2551 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[17385]: audit 2024-02-10T08:37:22.296412+0000 mgr.y (mgr.14547) 582 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[24488]: cluster 2024-02-10T08:37:22.189165+0000 mgr.y (mgr.14547) 581 : cluster [DBG] pgmap v881: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[24488]: cluster 2024-02-10T08:37:22.205339+0000 mon.a (mon.0) 2548 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[24488]: audit 2024-02-10T08:37:22.238082+0000 mon.a (mon.0) 2549 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[24488]: audit 2024-02-10T08:37:22.239219+0000 mon.a (mon.0) 2550 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[24488]: audit 2024-02-10T08:37:22.239663+0000 mon.a (mon.0) 2551 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:23.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:23 smithi067 bash[24488]: audit 2024-02-10T08:37:22.296412+0000 mgr.y (mgr.14547) 582 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:25.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:24 smithi134 bash[18722]: audit 2024-02-10T08:37:23.227464+0000 mon.a (mon.0) 2552 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:25.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:24 smithi134 bash[18722]: cluster 2024-02-10T08:37:23.227571+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-10T08:37:25.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:24 smithi134 bash[18722]: audit 2024-02-10T08:37:23.228782+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:24 smithi067 bash[17385]: audit 2024-02-10T08:37:23.227464+0000 mon.a (mon.0) 2552 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:24 smithi067 bash[17385]: cluster 2024-02-10T08:37:23.227571+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-10T08:37:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:24 smithi067 bash[17385]: audit 2024-02-10T08:37:23.228782+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:25.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:24 smithi067 bash[24488]: audit 2024-02-10T08:37:23.227464+0000 mon.a (mon.0) 2552 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:24 smithi067 bash[24488]: cluster 2024-02-10T08:37:23.227571+0000 mon.a (mon.0) 2553 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-10T08:37:25.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:24 smithi067 bash[24488]: audit 2024-02-10T08:37:23.228782+0000 mon.a (mon.0) 2554 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:26.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:25 smithi134 bash[18722]: cluster 2024-02-10T08:37:24.189958+0000 mgr.y (mgr.14547) 583 : cluster [DBG] pgmap v884: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:26.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:25 smithi134 bash[18722]: cluster 2024-02-10T08:37:24.679635+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-10T08:37:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:25 smithi134 bash[18722]: audit 2024-02-10T08:37:25.235082+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:26.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:25 smithi134 bash[18722]: cluster 2024-02-10T08:37:25.235296+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-10T08:37:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[17385]: cluster 2024-02-10T08:37:24.189958+0000 mgr.y (mgr.14547) 583 : cluster [DBG] pgmap v884: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[17385]: cluster 2024-02-10T08:37:24.679635+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-10T08:37:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[17385]: audit 2024-02-10T08:37:25.235082+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[17385]: cluster 2024-02-10T08:37:25.235296+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-10T08:37:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[24488]: cluster 2024-02-10T08:37:24.189958+0000 mgr.y (mgr.14547) 583 : cluster [DBG] pgmap v884: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[24488]: cluster 2024-02-10T08:37:24.679635+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-10T08:37:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[24488]: audit 2024-02-10T08:37:25.235082+0000 mon.a (mon.0) 2556 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:25 smithi067 bash[24488]: cluster 2024-02-10T08:37:25.235296+0000 mon.a (mon.0) 2557 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-10T08:37:27.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:27 smithi134 bash[18722]: cluster 2024-02-10T08:37:26.190698+0000 mgr.y (mgr.14547) 584 : cluster [DBG] pgmap v887: 180 pgs: 8 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:27.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:27 smithi134 bash[18722]: cluster 2024-02-10T08:37:26.241631+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-10T08:37:27.529 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:27 smithi134 bash[18722]: audit 2024-02-10T08:37:26.244190+0000 mon.a (mon.0) 2559 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[17385]: cluster 2024-02-10T08:37:26.190698+0000 mgr.y (mgr.14547) 584 : cluster [DBG] pgmap v887: 180 pgs: 8 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[17385]: cluster 2024-02-10T08:37:26.241631+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-10T08:37:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[17385]: audit 2024-02-10T08:37:26.244190+0000 mon.a (mon.0) 2559 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[24488]: cluster 2024-02-10T08:37:26.190698+0000 mgr.y (mgr.14547) 584 : cluster [DBG] pgmap v887: 180 pgs: 8 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[24488]: cluster 2024-02-10T08:37:26.241631+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-10T08:37:27.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[24488]: audit 2024-02-10T08:37:26.244190+0000 mon.a (mon.0) 2559 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:37:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:27] "GET /metrics HTTP/1.1" 200 192140 "" "Prometheus/2.33.4" 2024-02-10T08:37:28.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:28 smithi134 bash[18722]: audit 2024-02-10T08:37:27.241761+0000 mon.a (mon.0) 2560 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:28.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:28 smithi134 bash[18722]: cluster 2024-02-10T08:37:27.242028+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-10T08:37:28.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:28 smithi134 bash[18722]: audit 2024-02-10T08:37:27.256202+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:28 smithi067 bash[17385]: audit 2024-02-10T08:37:27.241761+0000 mon.a (mon.0) 2560 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:28 smithi067 bash[17385]: cluster 2024-02-10T08:37:27.242028+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-10T08:37:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:28 smithi067 bash[17385]: audit 2024-02-10T08:37:27.256202+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:28 smithi067 bash[24488]: audit 2024-02-10T08:37:27.241761+0000 mon.a (mon.0) 2560 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:28 smithi067 bash[24488]: cluster 2024-02-10T08:37:27.242028+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-10T08:37:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:28 smithi067 bash[24488]: audit 2024-02-10T08:37:27.256202+0000 mon.a (mon.0) 2562 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:29 smithi134 bash[18722]: cluster 2024-02-10T08:37:28.191632+0000 mgr.y (mgr.14547) 585 : cluster [DBG] pgmap v890: 212 pgs: 32 creating+peering, 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:29 smithi134 bash[18722]: audit 2024-02-10T08:37:28.262860+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:29 smithi134 bash[18722]: cluster 2024-02-10T08:37:28.262975+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-10T08:37:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:29 smithi134 bash[18722]: audit 2024-02-10T08:37:28.265756+0000 mon.a (mon.0) 2565 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-99", "overlaypool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[17385]: cluster 2024-02-10T08:37:28.191632+0000 mgr.y (mgr.14547) 585 : cluster [DBG] pgmap v890: 212 pgs: 32 creating+peering, 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[17385]: audit 2024-02-10T08:37:28.262860+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[17385]: cluster 2024-02-10T08:37:28.262975+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-10T08:37:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[17385]: audit 2024-02-10T08:37:28.265756+0000 mon.a (mon.0) 2565 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-99", "overlaypool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[24488]: cluster 2024-02-10T08:37:28.191632+0000 mgr.y (mgr.14547) 585 : cluster [DBG] pgmap v890: 212 pgs: 32 creating+peering, 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[24488]: audit 2024-02-10T08:37:28.262860+0000 mon.a (mon.0) 2563 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[24488]: cluster 2024-02-10T08:37:28.262975+0000 mon.a (mon.0) 2564 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-10T08:37:29.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:29 smithi067 bash[24488]: audit 2024-02-10T08:37:28.265756+0000 mon.a (mon.0) 2565 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-99", "overlaypool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: audit 2024-02-10T08:37:29.267574+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-99", "overlaypool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: cluster 2024-02-10T08:37:29.267703+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-10T08:37:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: audit 2024-02-10T08:37:29.268999+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-99-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:37:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: cluster 2024-02-10T08:37:30.262310+0000 mon.a (mon.0) 2569 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:30.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: audit 2024-02-10T08:37:30.269527+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-99-cache", "mode": "writeback"}]': finished 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: cluster 2024-02-10T08:37:30.269646+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[17385]: audit 2024-02-10T08:37:30.271010+0000 mon.a (mon.0) 2572 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: audit 2024-02-10T08:37:29.267574+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-99", "overlaypool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: cluster 2024-02-10T08:37:29.267703+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: audit 2024-02-10T08:37:29.268999+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-99-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: cluster 2024-02-10T08:37:30.262310+0000 mon.a (mon.0) 2569 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: audit 2024-02-10T08:37:30.269527+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-99-cache", "mode": "writeback"}]': finished 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: cluster 2024-02-10T08:37:30.269646+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-10T08:37:30.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:30 smithi067 bash[24488]: audit 2024-02-10T08:37:30.271010+0000 mon.a (mon.0) 2572 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:37:30.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: audit 2024-02-10T08:37:29.267574+0000 mon.a (mon.0) 2566 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-99", "overlaypool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:30.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: cluster 2024-02-10T08:37:29.267703+0000 mon.a (mon.0) 2567 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-10T08:37:30.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: audit 2024-02-10T08:37:29.268999+0000 mon.a (mon.0) 2568 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-99-cache", "mode": "writeback"}]: dispatch 2024-02-10T08:37:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: cluster 2024-02-10T08:37:30.262310+0000 mon.a (mon.0) 2569 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: audit 2024-02-10T08:37:30.269527+0000 mon.a (mon.0) 2570 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-99-cache", "mode": "writeback"}]': finished 2024-02-10T08:37:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: cluster 2024-02-10T08:37:30.269646+0000 mon.a (mon.0) 2571 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-10T08:37:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:30 smithi134 bash[18722]: audit 2024-02-10T08:37:30.271010+0000 mon.a (mon.0) 2572 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:37:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[24488]: cluster 2024-02-10T08:37:30.192644+0000 mgr.y (mgr.14547) 586 : cluster [DBG] pgmap v893: 212 pgs: 32 creating+peering, 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[24488]: audit 2024-02-10T08:37:30.788843+0000 mon.a (mon.0) 2573 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[24488]: audit 2024-02-10T08:37:31.273756+0000 mon.a (mon.0) 2574 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:37:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[24488]: cluster 2024-02-10T08:37:31.273855+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-10T08:37:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[24488]: audit 2024-02-10T08:37:31.275021+0000 mon.a (mon.0) 2576 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:37:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[17385]: cluster 2024-02-10T08:37:30.192644+0000 mgr.y (mgr.14547) 586 : cluster [DBG] pgmap v893: 212 pgs: 32 creating+peering, 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[17385]: audit 2024-02-10T08:37:30.788843+0000 mon.a (mon.0) 2573 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[17385]: audit 2024-02-10T08:37:31.273756+0000 mon.a (mon.0) 2574 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:37:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[17385]: cluster 2024-02-10T08:37:31.273855+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-10T08:37:32.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:31 smithi067 bash[17385]: audit 2024-02-10T08:37:31.275021+0000 mon.a (mon.0) 2576 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:37:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:31 smithi134 bash[18722]: cluster 2024-02-10T08:37:30.192644+0000 mgr.y (mgr.14547) 586 : cluster [DBG] pgmap v893: 212 pgs: 32 creating+peering, 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:31 smithi134 bash[18722]: audit 2024-02-10T08:37:30.788843+0000 mon.a (mon.0) 2573 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:31 smithi134 bash[18722]: audit 2024-02-10T08:37:31.273756+0000 mon.a (mon.0) 2574 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:37:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:31 smithi134 bash[18722]: cluster 2024-02-10T08:37:31.273855+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-10T08:37:32.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:31 smithi134 bash[18722]: audit 2024-02-10T08:37:31.275021+0000 mon.a (mon.0) 2576 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:37:32.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:37:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:37:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:37:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:37:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:33 smithi134 bash[18722]: cluster 2024-02-10T08:37:32.194926+0000 mgr.y (mgr.14547) 587 : cluster [DBG] pgmap v896: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:33 smithi134 bash[18722]: audit 2024-02-10T08:37:32.276998+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:37:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:33 smithi134 bash[18722]: cluster 2024-02-10T08:37:32.277097+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-10T08:37:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:33 smithi134 bash[18722]: audit 2024-02-10T08:37:32.278522+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:37:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:33 smithi134 bash[18722]: audit 2024-02-10T08:37:32.307175+0000 mgr.y (mgr.14547) 588 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[17385]: cluster 2024-02-10T08:37:32.194926+0000 mgr.y (mgr.14547) 587 : cluster [DBG] pgmap v896: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[17385]: audit 2024-02-10T08:37:32.276998+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[17385]: cluster 2024-02-10T08:37:32.277097+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[17385]: audit 2024-02-10T08:37:32.278522+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[17385]: audit 2024-02-10T08:37:32.307175+0000 mgr.y (mgr.14547) 588 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[24488]: cluster 2024-02-10T08:37:32.194926+0000 mgr.y (mgr.14547) 587 : cluster [DBG] pgmap v896: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[24488]: audit 2024-02-10T08:37:32.276998+0000 mon.a (mon.0) 2577 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[24488]: cluster 2024-02-10T08:37:32.277097+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[24488]: audit 2024-02-10T08:37:32.278522+0000 mon.a (mon.0) 2579 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:37:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:33 smithi067 bash[24488]: audit 2024-02-10T08:37:32.307175+0000 mgr.y (mgr.14547) 588 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[17385]: cluster 2024-02-10T08:37:33.271337+0000 mon.a (mon.0) 2580 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:37:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[17385]: audit 2024-02-10T08:37:33.284417+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:37:34.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[17385]: cluster 2024-02-10T08:37:33.284523+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-10T08:37:34.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[17385]: audit 2024-02-10T08:37:33.285304+0000 mon.a (mon.0) 2583 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T08:37:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[24488]: cluster 2024-02-10T08:37:33.271337+0000 mon.a (mon.0) 2580 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:37:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[24488]: audit 2024-02-10T08:37:33.284417+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:37:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[24488]: cluster 2024-02-10T08:37:33.284523+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-10T08:37:34.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:34 smithi067 bash[24488]: audit 2024-02-10T08:37:33.285304+0000 mon.a (mon.0) 2583 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T08:37:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:34 smithi134 bash[18722]: cluster 2024-02-10T08:37:33.271337+0000 mon.a (mon.0) 2580 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:37:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:34 smithi134 bash[18722]: audit 2024-02-10T08:37:33.284417+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:37:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:34 smithi134 bash[18722]: cluster 2024-02-10T08:37:33.284523+0000 mon.a (mon.0) 2582 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-10T08:37:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:34 smithi134 bash[18722]: audit 2024-02-10T08:37:33.285304+0000 mon.a (mon.0) 2583 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-10T08:37:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[17385]: cluster 2024-02-10T08:37:34.195682+0000 mgr.y (mgr.14547) 589 : cluster [DBG] pgmap v899: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[17385]: audit 2024-02-10T08:37:34.293454+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T08:37:35.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[17385]: cluster 2024-02-10T08:37:34.293516+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-10T08:37:35.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[17385]: audit 2024-02-10T08:37:34.357173+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[24488]: cluster 2024-02-10T08:37:34.195682+0000 mgr.y (mgr.14547) 589 : cluster [DBG] pgmap v899: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[24488]: audit 2024-02-10T08:37:34.293454+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T08:37:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[24488]: cluster 2024-02-10T08:37:34.293516+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-10T08:37:35.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:35 smithi067 bash[24488]: audit 2024-02-10T08:37:34.357173+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:35 smithi134 bash[18722]: cluster 2024-02-10T08:37:34.195682+0000 mgr.y (mgr.14547) 589 : cluster [DBG] pgmap v899: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:35 smithi134 bash[18722]: audit 2024-02-10T08:37:34.293454+0000 mon.a (mon.0) 2584 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-10T08:37:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:35 smithi134 bash[18722]: cluster 2024-02-10T08:37:34.293516+0000 mon.a (mon.0) 2585 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-10T08:37:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:35 smithi134 bash[18722]: audit 2024-02-10T08:37:34.357173+0000 mon.a (mon.0) 2586 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:36.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:36 smithi067 bash[24488]: audit 2024-02-10T08:37:35.301920+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:36.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:36 smithi067 bash[24488]: cluster 2024-02-10T08:37:35.302019+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-10T08:37:36.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:36 smithi067 bash[24488]: audit 2024-02-10T08:37:35.302759+0000 mon.a (mon.0) 2589 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:36.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:36 smithi067 bash[17385]: audit 2024-02-10T08:37:35.301920+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:36.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:36 smithi067 bash[17385]: cluster 2024-02-10T08:37:35.302019+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-10T08:37:36.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:36 smithi067 bash[17385]: audit 2024-02-10T08:37:35.302759+0000 mon.a (mon.0) 2589 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:36 smithi134 bash[18722]: audit 2024-02-10T08:37:35.301920+0000 mon.a (mon.0) 2587 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:36 smithi134 bash[18722]: cluster 2024-02-10T08:37:35.302019+0000 mon.a (mon.0) 2588 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-10T08:37:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:36 smithi134 bash[18722]: audit 2024-02-10T08:37:35.302759+0000 mon.a (mon.0) 2589 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]: dispatch 2024-02-10T08:37:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[17385]: cluster 2024-02-10T08:37:36.196525+0000 mgr.y (mgr.14547) 590 : cluster [DBG] pgmap v902: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[17385]: audit 2024-02-10T08:37:36.311279+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:37.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[17385]: cluster 2024-02-10T08:37:36.311388+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-10T08:37:37.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[24488]: cluster 2024-02-10T08:37:36.196525+0000 mgr.y (mgr.14547) 590 : cluster [DBG] pgmap v902: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:37.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[24488]: audit 2024-02-10T08:37:36.311279+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:37.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[24488]: cluster 2024-02-10T08:37:36.311388+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-10T08:37:37.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:37 smithi134 bash[18722]: cluster 2024-02-10T08:37:36.196525+0000 mgr.y (mgr.14547) 590 : cluster [DBG] pgmap v902: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:37.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:37 smithi134 bash[18722]: audit 2024-02-10T08:37:36.311279+0000 mon.a (mon.0) 2590 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-99", "tierpool": "test-rados-api-smithi067-54339-99-cache"}]': finished 2024-02-10T08:37:37.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:37 smithi134 bash[18722]: cluster 2024-02-10T08:37:36.311388+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-10T08:37:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:37:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:37] "GET /metrics HTTP/1.1" 200 192446 "" "Prometheus/2.33.4" 2024-02-10T08:37:38.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:38 smithi067 bash[17385]: cluster 2024-02-10T08:37:37.315443+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-10T08:37:38.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:38 smithi067 bash[17385]: cluster 2024-02-10T08:37:38.320139+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-10T08:37:38.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:38 smithi067 bash[17385]: audit 2024-02-10T08:37:38.321925+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:38 smithi067 bash[24488]: cluster 2024-02-10T08:37:37.315443+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-10T08:37:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:38 smithi067 bash[24488]: cluster 2024-02-10T08:37:38.320139+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-10T08:37:38.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:38 smithi067 bash[24488]: audit 2024-02-10T08:37:38.321925+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:38 smithi134 bash[18722]: cluster 2024-02-10T08:37:37.315443+0000 mon.a (mon.0) 2592 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-10T08:37:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:38 smithi134 bash[18722]: cluster 2024-02-10T08:37:38.320139+0000 mon.a (mon.0) 2593 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-10T08:37:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:38 smithi134 bash[18722]: audit 2024-02-10T08:37:38.321925+0000 mon.a (mon.0) 2594 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[17385]: cluster 2024-02-10T08:37:38.197471+0000 mgr.y (mgr.14547) 591 : cluster [DBG] pgmap v905: 180 pgs: 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[17385]: cluster 2024-02-10T08:37:38.323891+0000 mon.a (mon.0) 2595 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:39.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[17385]: audit 2024-02-10T08:37:39.321240+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[17385]: cluster 2024-02-10T08:37:39.321383+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[17385]: audit 2024-02-10T08:37:39.322233+0000 mon.a (mon.0) 2598 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[24488]: cluster 2024-02-10T08:37:38.197471+0000 mgr.y (mgr.14547) 591 : cluster [DBG] pgmap v905: 180 pgs: 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[24488]: cluster 2024-02-10T08:37:38.323891+0000 mon.a (mon.0) 2595 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[24488]: audit 2024-02-10T08:37:39.321240+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[24488]: cluster 2024-02-10T08:37:39.321383+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-10T08:37:39.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:39 smithi067 bash[24488]: audit 2024-02-10T08:37:39.322233+0000 mon.a (mon.0) 2598 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:39 smithi134 bash[18722]: cluster 2024-02-10T08:37:38.197471+0000 mgr.y (mgr.14547) 591 : cluster [DBG] pgmap v905: 180 pgs: 180 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:39 smithi134 bash[18722]: cluster 2024-02-10T08:37:38.323891+0000 mon.a (mon.0) 2595 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:39 smithi134 bash[18722]: audit 2024-02-10T08:37:39.321240+0000 mon.a (mon.0) 2596 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:39 smithi134 bash[18722]: cluster 2024-02-10T08:37:39.321383+0000 mon.a (mon.0) 2597 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-10T08:37:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:39 smithi134 bash[18722]: audit 2024-02-10T08:37:39.322233+0000 mon.a (mon.0) 2598 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]: dispatch 2024-02-10T08:37:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:41 smithi067 bash[17385]: cluster 2024-02-10T08:37:40.198299+0000 mgr.y (mgr.14547) 592 : cluster [DBG] pgmap v908: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:41 smithi067 bash[17385]: audit 2024-02-10T08:37:40.324000+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:41 smithi067 bash[17385]: cluster 2024-02-10T08:37:40.324135+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-10T08:37:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:41 smithi067 bash[24488]: cluster 2024-02-10T08:37:40.198299+0000 mgr.y (mgr.14547) 592 : cluster [DBG] pgmap v908: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:41 smithi067 bash[24488]: audit 2024-02-10T08:37:40.324000+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:41.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:41 smithi067 bash[24488]: cluster 2024-02-10T08:37:40.324135+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-10T08:37:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:41 smithi134 bash[18722]: cluster 2024-02-10T08:37:40.198299+0000 mgr.y (mgr.14547) 592 : cluster [DBG] pgmap v908: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:41 smithi134 bash[18722]: audit 2024-02-10T08:37:40.324000+0000 mon.a (mon.0) 2599 : audit [INF] from='client.? 172.21.15.67:0/954738253' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-99"}]': finished 2024-02-10T08:37:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:41 smithi134 bash[18722]: cluster 2024-02-10T08:37:40.324135+0000 mon.a (mon.0) 2600 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-10T08:37:42.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[24488]: cluster 2024-02-10T08:37:41.328272+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-10T08:37:42.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[24488]: audit 2024-02-10T08:37:41.330641+0000 mon.a (mon.0) 2602 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:42.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[24488]: audit 2024-02-10T08:37:42.330725+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:42.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[24488]: cluster 2024-02-10T08:37:42.330891+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-10T08:37:42.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[24488]: audit 2024-02-10T08:37:42.331774+0000 mon.a (mon.0) 2605 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[17385]: cluster 2024-02-10T08:37:41.328272+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-10T08:37:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[17385]: audit 2024-02-10T08:37:41.330641+0000 mon.a (mon.0) 2602 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[17385]: audit 2024-02-10T08:37:42.330725+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[17385]: cluster 2024-02-10T08:37:42.330891+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-10T08:37:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:42 smithi067 bash[17385]: audit 2024-02-10T08:37:42.331774+0000 mon.a (mon.0) 2605 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:42.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:37:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:37:42.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:42 smithi134 bash[18722]: cluster 2024-02-10T08:37:41.328272+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-10T08:37:42.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:42 smithi134 bash[18722]: audit 2024-02-10T08:37:41.330641+0000 mon.a (mon.0) 2602 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:42 smithi134 bash[18722]: audit 2024-02-10T08:37:42.330725+0000 mon.a (mon.0) 2603 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:42 smithi134 bash[18722]: cluster 2024-02-10T08:37:42.330891+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-10T08:37:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:42 smithi134 bash[18722]: audit 2024-02-10T08:37:42.331774+0000 mon.a (mon.0) 2605 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]: dispatch 2024-02-10T08:37:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:37:44.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: cluster 2024-02-10T08:37:42.199899+0000 mgr.y (mgr.14547) 593 : cluster [DBG] pgmap v911: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:37:44.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:42.318455+0000 mgr.y (mgr.14547) 594 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.333306+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: cluster 2024-02-10T08:37:43.333409+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.358966+0000 mon.c (mon.2) 869 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.359533+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.360605+0000 mon.c (mon.2) 870 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.361131+0000 mon.a (mon.0) 2609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.361937+0000 mon.c (mon.2) 871 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:43 smithi134 bash[18722]: audit 2024-02-10T08:37:43.362442+0000 mon.a (mon.0) 2610 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: cluster 2024-02-10T08:37:42.199899+0000 mgr.y (mgr.14547) 593 : cluster [DBG] pgmap v911: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:37:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:42.318455+0000 mgr.y (mgr.14547) 594 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.333306+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: cluster 2024-02-10T08:37:43.333409+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.358966+0000 mon.c (mon.2) 869 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.359533+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.360605+0000 mon.c (mon.2) 870 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.361131+0000 mon.a (mon.0) 2609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.361937+0000 mon.c (mon.2) 871 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[17385]: audit 2024-02-10T08:37:43.362442+0000 mon.a (mon.0) 2610 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: cluster 2024-02-10T08:37:42.199899+0000 mgr.y (mgr.14547) 593 : cluster [DBG] pgmap v911: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:42.318455+0000 mgr.y (mgr.14547) 594 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:44.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.333306+0000 mon.a (mon.0) 2606 : audit [INF] from='client.? 172.21.15.67:0/864368300' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi067-54339-94"}]': finished 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: cluster 2024-02-10T08:37:43.333409+0000 mon.a (mon.0) 2607 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.358966+0000 mon.c (mon.2) 869 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.359533+0000 mon.a (mon.0) 2608 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.360605+0000 mon.c (mon.2) 870 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.361131+0000 mon.a (mon.0) 2609 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.361937+0000 mon.c (mon.2) 871 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:44.122 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:43 smithi067 bash[24488]: audit 2024-02-10T08:37:43.362442+0000 mon.a (mon.0) 2610 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-10T08:37:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[17385]: cluster 2024-02-10T08:37:44.200719+0000 mgr.y (mgr.14547) 595 : cluster [DBG] pgmap v914: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:37:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[17385]: audit 2024-02-10T08:37:44.336676+0000 mon.a (mon.0) 2611 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[17385]: cluster 2024-02-10T08:37:44.336806+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[17385]: audit 2024-02-10T08:37:44.339437+0000 mon.c (mon.2) 872 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[17385]: audit 2024-02-10T08:37:44.340050+0000 mon.a (mon.0) 2613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[24488]: cluster 2024-02-10T08:37:44.200719+0000 mgr.y (mgr.14547) 595 : cluster [DBG] pgmap v914: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[24488]: audit 2024-02-10T08:37:44.336676+0000 mon.a (mon.0) 2611 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[24488]: cluster 2024-02-10T08:37:44.336806+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[24488]: audit 2024-02-10T08:37:44.339437+0000 mon.c (mon.2) 872 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:45 smithi067 bash[24488]: audit 2024-02-10T08:37:44.340050+0000 mon.a (mon.0) 2613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:46.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:45 smithi134 bash[18722]: cluster 2024-02-10T08:37:44.200719+0000 mgr.y (mgr.14547) 595 : cluster [DBG] pgmap v914: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-10T08:37:46.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:45 smithi134 bash[18722]: audit 2024-02-10T08:37:44.336676+0000 mon.a (mon.0) 2611 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi067-54339-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-10T08:37:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:45 smithi134 bash[18722]: cluster 2024-02-10T08:37:44.336806+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-10T08:37:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:45 smithi134 bash[18722]: audit 2024-02-10T08:37:44.339437+0000 mon.c (mon.2) 872 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:46.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:45 smithi134 bash[18722]: audit 2024-02-10T08:37:44.340050+0000 mon.a (mon.0) 2613 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[17385]: cluster 2024-02-10T08:37:45.790556+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-10T08:37:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[17385]: audit 2024-02-10T08:37:45.802978+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[17385]: audit 2024-02-10T08:37:46.345495+0000 mon.a (mon.0) 2616 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:37:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[17385]: cluster 2024-02-10T08:37:46.345726+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-10T08:37:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[24488]: cluster 2024-02-10T08:37:45.790556+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-10T08:37:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[24488]: audit 2024-02-10T08:37:45.802978+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[24488]: audit 2024-02-10T08:37:46.345495+0000 mon.a (mon.0) 2616 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:37:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:46 smithi067 bash[24488]: cluster 2024-02-10T08:37:46.345726+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-10T08:37:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:46 smithi134 bash[18722]: cluster 2024-02-10T08:37:45.790556+0000 mon.a (mon.0) 2614 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-10T08:37:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:46 smithi134 bash[18722]: audit 2024-02-10T08:37:45.802978+0000 mon.a (mon.0) 2615 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:46 smithi134 bash[18722]: audit 2024-02-10T08:37:46.345495+0000 mon.a (mon.0) 2616 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi067-54339-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:37:47.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:46 smithi134 bash[18722]: cluster 2024-02-10T08:37:46.345726+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-10T08:37:48.122 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:47] "GET /metrics HTTP/1.1" 200 185173 "" "Prometheus/2.33.4" 2024-02-10T08:37:48.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[17385]: cluster 2024-02-10T08:37:46.201513+0000 mgr.y (mgr.14547) 596 : cluster [DBG] pgmap v917: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:48.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[17385]: cluster 2024-02-10T08:37:46.795043+0000 mon.a (mon.0) 2618 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:48.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[17385]: cluster 2024-02-10T08:37:47.347679+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-10T08:37:48.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[17385]: audit 2024-02-10T08:37:47.349413+0000 mon.c (mon.2) 873 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:48.122 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[17385]: audit 2024-02-10T08:37:47.350048+0000 mon.a (mon.0) 2620 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[24488]: cluster 2024-02-10T08:37:46.201513+0000 mgr.y (mgr.14547) 596 : cluster [DBG] pgmap v917: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[24488]: cluster 2024-02-10T08:37:46.795043+0000 mon.a (mon.0) 2618 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[24488]: cluster 2024-02-10T08:37:47.347679+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-10T08:37:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[24488]: audit 2024-02-10T08:37:47.349413+0000 mon.c (mon.2) 873 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:48.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:47 smithi067 bash[24488]: audit 2024-02-10T08:37:47.350048+0000 mon.a (mon.0) 2620 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:48.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:47 smithi134 bash[18722]: cluster 2024-02-10T08:37:46.201513+0000 mgr.y (mgr.14547) 596 : cluster [DBG] pgmap v917: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:47 smithi134 bash[18722]: cluster 2024-02-10T08:37:46.795043+0000 mon.a (mon.0) 2618 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:47 smithi134 bash[18722]: cluster 2024-02-10T08:37:47.347679+0000 mon.a (mon.0) 2619 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-10T08:37:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:47 smithi134 bash[18722]: audit 2024-02-10T08:37:47.349413+0000 mon.c (mon.2) 873 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:48.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:47 smithi134 bash[18722]: audit 2024-02-10T08:37:47.350048+0000 mon.a (mon.0) 2620 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[17385]: cluster 2024-02-10T08:37:48.202450+0000 mgr.y (mgr.14547) 597 : cluster [DBG] pgmap v920: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[17385]: audit 2024-02-10T08:37:48.350525+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[17385]: cluster 2024-02-10T08:37:48.350967+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-10T08:37:49.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[17385]: audit 2024-02-10T08:37:48.376393+0000 mon.c (mon.2) 874 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:49.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[17385]: audit 2024-02-10T08:37:48.376743+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[24488]: cluster 2024-02-10T08:37:48.202450+0000 mgr.y (mgr.14547) 597 : cluster [DBG] pgmap v920: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[24488]: audit 2024-02-10T08:37:48.350525+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[24488]: cluster 2024-02-10T08:37:48.350967+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-10T08:37:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[24488]: audit 2024-02-10T08:37:48.376393+0000 mon.c (mon.2) 874 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:49.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:49 smithi067 bash[24488]: audit 2024-02-10T08:37:48.376743+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:49 smithi134 bash[18722]: cluster 2024-02-10T08:37:48.202450+0000 mgr.y (mgr.14547) 597 : cluster [DBG] pgmap v920: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:49 smithi134 bash[18722]: audit 2024-02-10T08:37:48.350525+0000 mon.a (mon.0) 2621 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:49 smithi134 bash[18722]: cluster 2024-02-10T08:37:48.350967+0000 mon.a (mon.0) 2622 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-10T08:37:49.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:49 smithi134 bash[18722]: audit 2024-02-10T08:37:48.376393+0000 mon.c (mon.2) 874 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:49.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:49 smithi134 bash[18722]: audit 2024-02-10T08:37:48.376743+0000 mon.a (mon.0) 2623 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[17385]: audit 2024-02-10T08:37:49.353005+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:37:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[17385]: cluster 2024-02-10T08:37:49.353171+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-10T08:37:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[17385]: audit 2024-02-10T08:37:49.354253+0000 mon.c (mon.2) 875 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:50.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[17385]: audit 2024-02-10T08:37:49.355569+0000 mon.a (mon.0) 2626 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[17385]: audit 2024-02-10T08:37:50.355386+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]': finished 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[17385]: cluster 2024-02-10T08:37:50.355544+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[24488]: audit 2024-02-10T08:37:49.353005+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[24488]: cluster 2024-02-10T08:37:49.353171+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[24488]: audit 2024-02-10T08:37:49.354253+0000 mon.c (mon.2) 875 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[24488]: audit 2024-02-10T08:37:49.355569+0000 mon.a (mon.0) 2626 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[24488]: audit 2024-02-10T08:37:50.355386+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]': finished 2024-02-10T08:37:50.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:50 smithi067 bash[24488]: cluster 2024-02-10T08:37:50.355544+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-10T08:37:50.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:50 smithi134 bash[18722]: audit 2024-02-10T08:37:49.353005+0000 mon.a (mon.0) 2624 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:37:50.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:50 smithi134 bash[18722]: cluster 2024-02-10T08:37:49.353171+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-10T08:37:50.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:50 smithi134 bash[18722]: audit 2024-02-10T08:37:49.354253+0000 mon.c (mon.2) 875 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:50 smithi134 bash[18722]: audit 2024-02-10T08:37:49.355569+0000 mon.a (mon.0) 2626 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:50 smithi134 bash[18722]: audit 2024-02-10T08:37:50.355386+0000 mon.a (mon.0) 2627 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-102"}]': finished 2024-02-10T08:37:50.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:50 smithi134 bash[18722]: cluster 2024-02-10T08:37:50.355544+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-10T08:37:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:51 smithi067 bash[17385]: cluster 2024-02-10T08:37:50.203253+0000 mgr.y (mgr.14547) 598 : cluster [DBG] pgmap v923: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:51 smithi067 bash[17385]: audit 2024-02-10T08:37:50.405440+0000 mon.c (mon.2) 876 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:51.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:51 smithi067 bash[17385]: audit 2024-02-10T08:37:50.406136+0000 mon.a (mon.0) 2629 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:51.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:51 smithi067 bash[24488]: cluster 2024-02-10T08:37:50.203253+0000 mgr.y (mgr.14547) 598 : cluster [DBG] pgmap v923: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:51 smithi067 bash[24488]: audit 2024-02-10T08:37:50.405440+0000 mon.c (mon.2) 876 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:51.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:51 smithi067 bash[24488]: audit 2024-02-10T08:37:50.406136+0000 mon.a (mon.0) 2629 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:51 smithi134 bash[18722]: cluster 2024-02-10T08:37:50.203253+0000 mgr.y (mgr.14547) 598 : cluster [DBG] pgmap v923: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:37:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:51 smithi134 bash[18722]: audit 2024-02-10T08:37:50.405440+0000 mon.c (mon.2) 876 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:51.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:51 smithi134 bash[18722]: audit 2024-02-10T08:37:50.406136+0000 mon.a (mon.0) 2629 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:52.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:52 smithi134 bash[18722]: audit 2024-02-10T08:37:51.378567+0000 mon.a (mon.0) 2630 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:37:52.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:52 smithi134 bash[18722]: cluster 2024-02-10T08:37:51.378689+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-10T08:37:52.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:52 smithi134 bash[18722]: audit 2024-02-10T08:37:51.382863+0000 mon.c (mon.2) 877 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:52.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:52 smithi134 bash[18722]: audit 2024-02-10T08:37:51.383414+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:52.777 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:37:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:37:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[17385]: audit 2024-02-10T08:37:51.378567+0000 mon.a (mon.0) 2630 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:37:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[17385]: cluster 2024-02-10T08:37:51.378689+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-10T08:37:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[17385]: audit 2024-02-10T08:37:51.382863+0000 mon.c (mon.2) 877 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[17385]: audit 2024-02-10T08:37:51.383414+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:52.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[24488]: audit 2024-02-10T08:37:51.378567+0000 mon.a (mon.0) 2630 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:37:52.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[24488]: cluster 2024-02-10T08:37:51.378689+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-10T08:37:52.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[24488]: audit 2024-02-10T08:37:51.382863+0000 mon.c (mon.2) 877 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:52.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:52 smithi067 bash[24488]: audit 2024-02-10T08:37:51.383414+0000 mon.a (mon.0) 2632 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]: dispatch 2024-02-10T08:37:53.405 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:37:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:37:53.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:53 smithi134 bash[18722]: cluster 2024-02-10T08:37:52.205287+0000 mgr.y (mgr.14547) 599 : cluster [DBG] pgmap v926: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:37:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:53 smithi134 bash[18722]: audit 2024-02-10T08:37:52.329439+0000 mgr.y (mgr.14547) 600 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:53 smithi134 bash[18722]: audit 2024-02-10T08:37:52.383465+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]': finished 2024-02-10T08:37:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:53 smithi134 bash[18722]: cluster 2024-02-10T08:37:52.383700+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-10T08:37:53.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:53 smithi134 bash[18722]: cluster 2024-02-10T08:37:52.822095+0000 mon.a (mon.0) 2635 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[17385]: cluster 2024-02-10T08:37:52.205287+0000 mgr.y (mgr.14547) 599 : cluster [DBG] pgmap v926: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:37:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[17385]: audit 2024-02-10T08:37:52.329439+0000 mgr.y (mgr.14547) 600 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[17385]: audit 2024-02-10T08:37:52.383465+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]': finished 2024-02-10T08:37:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[17385]: cluster 2024-02-10T08:37:52.383700+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-10T08:37:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[17385]: cluster 2024-02-10T08:37:52.822095+0000 mon.a (mon.0) 2635 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[24488]: cluster 2024-02-10T08:37:52.205287+0000 mgr.y (mgr.14547) 599 : cluster [DBG] pgmap v926: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:37:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[24488]: audit 2024-02-10T08:37:52.329439+0000 mgr.y (mgr.14547) 600 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:37:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[24488]: audit 2024-02-10T08:37:52.383465+0000 mon.a (mon.0) 2633 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-102"}]': finished 2024-02-10T08:37:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[24488]: cluster 2024-02-10T08:37:52.383700+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-10T08:37:53.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:53 smithi067 bash[24488]: cluster 2024-02-10T08:37:52.822095+0000 mon.a (mon.0) 2635 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:37:54.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:54 smithi134 bash[18722]: cluster 2024-02-10T08:37:53.397549+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-10T08:37:54.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:54 smithi067 bash[17385]: cluster 2024-02-10T08:37:53.397549+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-10T08:37:54.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:54 smithi067 bash[24488]: cluster 2024-02-10T08:37:53.397549+0000 mon.a (mon.0) 2636 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-10T08:37:55.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: cluster 2024-02-10T08:37:54.206120+0000 mgr.y (mgr.14547) 601 : cluster [DBG] pgmap v929: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:55.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: cluster 2024-02-10T08:37:54.420296+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-10T08:37:55.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: audit 2024-02-10T08:37:54.430305+0000 mon.c (mon.2) 878 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: audit 2024-02-10T08:37:54.433121+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: audit 2024-02-10T08:37:54.609823+0000 mon.c (mon.2) 879 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:37:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: audit 2024-02-10T08:37:54.917741+0000 mon.c (mon.2) 880 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:37:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: audit 2024-02-10T08:37:54.919016+0000 mon.c (mon.2) 881 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:37:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:55 smithi134 bash[18722]: audit 2024-02-10T08:37:54.926384+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: cluster 2024-02-10T08:37:54.206120+0000 mgr.y (mgr.14547) 601 : cluster [DBG] pgmap v929: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: cluster 2024-02-10T08:37:54.420296+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-10T08:37:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: audit 2024-02-10T08:37:54.430305+0000 mon.c (mon.2) 878 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: audit 2024-02-10T08:37:54.433121+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: audit 2024-02-10T08:37:54.609823+0000 mon.c (mon.2) 879 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: audit 2024-02-10T08:37:54.917741+0000 mon.c (mon.2) 880 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: audit 2024-02-10T08:37:54.919016+0000 mon.c (mon.2) 881 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[17385]: audit 2024-02-10T08:37:54.926384+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: cluster 2024-02-10T08:37:54.206120+0000 mgr.y (mgr.14547) 601 : cluster [DBG] pgmap v929: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: cluster 2024-02-10T08:37:54.420296+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: audit 2024-02-10T08:37:54.430305+0000 mon.c (mon.2) 878 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: audit 2024-02-10T08:37:54.433121+0000 mon.a (mon.0) 2638 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: audit 2024-02-10T08:37:54.609823+0000 mon.c (mon.2) 879 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: audit 2024-02-10T08:37:54.917741+0000 mon.c (mon.2) 880 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: audit 2024-02-10T08:37:54.919016+0000 mon.c (mon.2) 881 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:37:55.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:55 smithi067 bash[24488]: audit 2024-02-10T08:37:54.926384+0000 mon.a (mon.0) 2639 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:37:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:56 smithi134 bash[18722]: audit 2024-02-10T08:37:55.428160+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:56 smithi134 bash[18722]: cluster 2024-02-10T08:37:55.428340+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-10T08:37:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:56 smithi134 bash[18722]: audit 2024-02-10T08:37:55.455375+0000 mon.c (mon.2) 882 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:56 smithi134 bash[18722]: audit 2024-02-10T08:37:55.455995+0000 mon.a (mon.0) 2642 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[17385]: audit 2024-02-10T08:37:55.428160+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[17385]: cluster 2024-02-10T08:37:55.428340+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-10T08:37:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[17385]: audit 2024-02-10T08:37:55.455375+0000 mon.c (mon.2) 882 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[17385]: audit 2024-02-10T08:37:55.455995+0000 mon.a (mon.0) 2642 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[24488]: audit 2024-02-10T08:37:55.428160+0000 mon.a (mon.0) 2640 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:37:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[24488]: cluster 2024-02-10T08:37:55.428340+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-10T08:37:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[24488]: audit 2024-02-10T08:37:55.455375+0000 mon.c (mon.2) 882 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:56 smithi067 bash[24488]: audit 2024-02-10T08:37:55.455995+0000 mon.a (mon.0) 2642 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:37:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:57 smithi134 bash[18722]: cluster 2024-02-10T08:37:56.206913+0000 mgr.y (mgr.14547) 602 : cluster [DBG] pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:57 smithi134 bash[18722]: audit 2024-02-10T08:37:56.454005+0000 mon.a (mon.0) 2643 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:37:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:57 smithi134 bash[18722]: cluster 2024-02-10T08:37:56.454181+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-10T08:37:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:57 smithi134 bash[18722]: audit 2024-02-10T08:37:56.456495+0000 mon.c (mon.2) 883 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:37:57.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:57 smithi134 bash[18722]: audit 2024-02-10T08:37:56.458255+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:37:57.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[24488]: cluster 2024-02-10T08:37:56.206913+0000 mgr.y (mgr.14547) 602 : cluster [DBG] pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:57.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[24488]: audit 2024-02-10T08:37:56.454005+0000 mon.a (mon.0) 2643 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:37:57.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[24488]: cluster 2024-02-10T08:37:56.454181+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[24488]: audit 2024-02-10T08:37:56.456495+0000 mon.c (mon.2) 883 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[24488]: audit 2024-02-10T08:37:56.458255+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[17385]: cluster 2024-02-10T08:37:56.206913+0000 mgr.y (mgr.14547) 602 : cluster [DBG] pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[17385]: audit 2024-02-10T08:37:56.454005+0000 mon.a (mon.0) 2643 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[17385]: cluster 2024-02-10T08:37:56.454181+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[17385]: audit 2024-02-10T08:37:56.456495+0000 mon.c (mon.2) 883 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:37:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[17385]: audit 2024-02-10T08:37:56.458255+0000 mon.a (mon.0) 2645 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:37:58.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:37:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:37:57] "GET /metrics HTTP/1.1" 200 185173 "" "Prometheus/2.33.4" 2024-02-10T08:37:58.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:58 smithi134 bash[18722]: audit 2024-02-10T08:37:57.456914+0000 mon.a (mon.0) 2646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]': finished 2024-02-10T08:37:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:58 smithi134 bash[18722]: cluster 2024-02-10T08:37:57.457198+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-10T08:37:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:58 smithi134 bash[18722]: audit 2024-02-10T08:37:57.458188+0000 mon.c (mon.2) 884 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]: dispatch 2024-02-10T08:37:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:58 smithi134 bash[18722]: audit 2024-02-10T08:37:57.458995+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]: dispatch 2024-02-10T08:37:58.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:58 smithi134 bash[18722]: cluster 2024-02-10T08:37:58.450380+0000 mon.a (mon.0) 2649 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[17385]: audit 2024-02-10T08:37:57.456914+0000 mon.a (mon.0) 2646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]': finished 2024-02-10T08:37:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[17385]: cluster 2024-02-10T08:37:57.457198+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-10T08:37:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[17385]: audit 2024-02-10T08:37:57.458188+0000 mon.c (mon.2) 884 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]: dispatch 2024-02-10T08:37:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[17385]: audit 2024-02-10T08:37:57.458995+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]: dispatch 2024-02-10T08:37:58.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[17385]: cluster 2024-02-10T08:37:58.450380+0000 mon.a (mon.0) 2649 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[24488]: audit 2024-02-10T08:37:57.456914+0000 mon.a (mon.0) 2646 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-104"}]': finished 2024-02-10T08:37:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[24488]: cluster 2024-02-10T08:37:57.457198+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-10T08:37:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[24488]: audit 2024-02-10T08:37:57.458188+0000 mon.c (mon.2) 884 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]: dispatch 2024-02-10T08:37:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[24488]: audit 2024-02-10T08:37:57.458995+0000 mon.a (mon.0) 2648 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]: dispatch 2024-02-10T08:37:58.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:58 smithi067 bash[24488]: cluster 2024-02-10T08:37:58.450380+0000 mon.a (mon.0) 2649 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:37:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:59 smithi134 bash[18722]: cluster 2024-02-10T08:37:58.207780+0000 mgr.y (mgr.14547) 603 : cluster [DBG] pgmap v935: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:59 smithi134 bash[18722]: audit 2024-02-10T08:37:58.459254+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]': finished 2024-02-10T08:37:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:59 smithi134 bash[18722]: cluster 2024-02-10T08:37:58.459403+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-10T08:37:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:59 smithi134 bash[18722]: audit 2024-02-10T08:37:58.535844+0000 mon.c (mon.2) 885 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:37:59 smithi134 bash[18722]: audit 2024-02-10T08:37:58.536497+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[17385]: cluster 2024-02-10T08:37:58.207780+0000 mgr.y (mgr.14547) 603 : cluster [DBG] pgmap v935: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[17385]: audit 2024-02-10T08:37:58.459254+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]': finished 2024-02-10T08:37:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[17385]: cluster 2024-02-10T08:37:58.459403+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-10T08:37:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[17385]: audit 2024-02-10T08:37:58.535844+0000 mon.c (mon.2) 885 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:59.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[17385]: audit 2024-02-10T08:37:58.536497+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[24488]: cluster 2024-02-10T08:37:58.207780+0000 mgr.y (mgr.14547) 603 : cluster [DBG] pgmap v935: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:37:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[24488]: audit 2024-02-10T08:37:58.459254+0000 mon.a (mon.0) 2650 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-104", "mode": "writeback"}]': finished 2024-02-10T08:37:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[24488]: cluster 2024-02-10T08:37:58.459403+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-10T08:37:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[24488]: audit 2024-02-10T08:37:58.535844+0000 mon.c (mon.2) 885 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:37:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:37:59 smithi067 bash[24488]: audit 2024-02-10T08:37:58.536497+0000 mon.a (mon.0) 2652 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:00 smithi134 bash[18722]: audit 2024-02-10T08:37:59.486090+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:00 smithi134 bash[18722]: cluster 2024-02-10T08:37:59.486207+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-10T08:38:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:00 smithi134 bash[18722]: audit 2024-02-10T08:37:59.488777+0000 mon.c (mon.2) 886 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:38:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:00 smithi134 bash[18722]: audit 2024-02-10T08:37:59.489229+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:38:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[17385]: audit 2024-02-10T08:37:59.486090+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[17385]: cluster 2024-02-10T08:37:59.486207+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-10T08:38:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[17385]: audit 2024-02-10T08:37:59.488777+0000 mon.c (mon.2) 886 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:38:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[17385]: audit 2024-02-10T08:37:59.489229+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:38:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[24488]: audit 2024-02-10T08:37:59.486090+0000 mon.a (mon.0) 2653 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[24488]: cluster 2024-02-10T08:37:59.486207+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-10T08:38:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[24488]: audit 2024-02-10T08:37:59.488777+0000 mon.c (mon.2) 886 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:38:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:00 smithi067 bash[24488]: audit 2024-02-10T08:37:59.489229+0000 mon.a (mon.0) 2655 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]: dispatch 2024-02-10T08:38:01.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:01 smithi134 bash[18722]: cluster 2024-02-10T08:38:00.208603+0000 mgr.y (mgr.14547) 604 : cluster [DBG] pgmap v938: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:01.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:01 smithi134 bash[18722]: cluster 2024-02-10T08:38:00.483187+0000 mon.a (mon.0) 2656 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:01.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:01 smithi134 bash[18722]: audit 2024-02-10T08:38:00.489291+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]': finished 2024-02-10T08:38:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:01 smithi134 bash[18722]: cluster 2024-02-10T08:38:00.489497+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-10T08:38:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[17385]: cluster 2024-02-10T08:38:00.208603+0000 mgr.y (mgr.14547) 604 : cluster [DBG] pgmap v938: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[17385]: cluster 2024-02-10T08:38:00.483187+0000 mon.a (mon.0) 2656 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[17385]: audit 2024-02-10T08:38:00.489291+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]': finished 2024-02-10T08:38:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[17385]: cluster 2024-02-10T08:38:00.489497+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-10T08:38:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[24488]: cluster 2024-02-10T08:38:00.208603+0000 mgr.y (mgr.14547) 604 : cluster [DBG] pgmap v938: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[24488]: cluster 2024-02-10T08:38:00.483187+0000 mon.a (mon.0) 2656 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[24488]: audit 2024-02-10T08:38:00.489291+0000 mon.a (mon.0) 2657 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-104"}]': finished 2024-02-10T08:38:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:01 smithi067 bash[24488]: cluster 2024-02-10T08:38:00.489497+0000 mon.a (mon.0) 2658 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-10T08:38:02.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:02 smithi134 bash[18722]: cluster 2024-02-10T08:38:01.514564+0000 mon.a (mon.0) 2659 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-10T08:38:02.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:38:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:38:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:02 smithi067 bash[17385]: cluster 2024-02-10T08:38:01.514564+0000 mon.a (mon.0) 2659 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-10T08:38:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:02 smithi067 bash[24488]: cluster 2024-02-10T08:38:01.514564+0000 mon.a (mon.0) 2659 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-10T08:38:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:38:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:38:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[17385]: cluster 2024-02-10T08:38:02.210360+0000 mgr.y (mgr.14547) 605 : cluster [DBG] pgmap v941: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[17385]: audit 2024-02-10T08:38:02.339070+0000 mgr.y (mgr.14547) 606 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[17385]: cluster 2024-02-10T08:38:02.522465+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-10T08:38:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[17385]: audit 2024-02-10T08:38:02.528506+0000 mon.c (mon.2) 887 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[17385]: audit 2024-02-10T08:38:02.530248+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[24488]: cluster 2024-02-10T08:38:02.210360+0000 mgr.y (mgr.14547) 605 : cluster [DBG] pgmap v941: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[24488]: audit 2024-02-10T08:38:02.339070+0000 mgr.y (mgr.14547) 606 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[24488]: cluster 2024-02-10T08:38:02.522465+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-10T08:38:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[24488]: audit 2024-02-10T08:38:02.528506+0000 mon.c (mon.2) 887 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:03 smithi067 bash[24488]: audit 2024-02-10T08:38:02.530248+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:03 smithi134 bash[18722]: cluster 2024-02-10T08:38:02.210360+0000 mgr.y (mgr.14547) 605 : cluster [DBG] pgmap v941: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:03 smithi134 bash[18722]: audit 2024-02-10T08:38:02.339070+0000 mgr.y (mgr.14547) 606 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:03 smithi134 bash[18722]: cluster 2024-02-10T08:38:02.522465+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-10T08:38:04.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:03 smithi134 bash[18722]: audit 2024-02-10T08:38:02.528506+0000 mon.c (mon.2) 887 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:04.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:03 smithi134 bash[18722]: audit 2024-02-10T08:38:02.530248+0000 mon.a (mon.0) 2661 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:04.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:04 smithi067 bash[17385]: audit 2024-02-10T08:38:03.528773+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:04.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:04 smithi067 bash[17385]: cluster 2024-02-10T08:38:03.529081+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-10T08:38:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:04 smithi067 bash[24488]: audit 2024-02-10T08:38:03.528773+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:04 smithi067 bash[24488]: cluster 2024-02-10T08:38:03.529081+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-10T08:38:05.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:04 smithi134 bash[18722]: audit 2024-02-10T08:38:03.528773+0000 mon.a (mon.0) 2662 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:05.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:04 smithi134 bash[18722]: cluster 2024-02-10T08:38:03.529081+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-10T08:38:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[17385]: cluster 2024-02-10T08:38:04.211210+0000 mgr.y (mgr.14547) 607 : cluster [DBG] pgmap v944: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[17385]: cluster 2024-02-10T08:38:04.546932+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-10T08:38:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[17385]: audit 2024-02-10T08:38:04.568146+0000 mon.c (mon.2) 888 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:05.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[17385]: audit 2024-02-10T08:38:04.568825+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[24488]: cluster 2024-02-10T08:38:04.211210+0000 mgr.y (mgr.14547) 607 : cluster [DBG] pgmap v944: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[24488]: cluster 2024-02-10T08:38:04.546932+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-10T08:38:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[24488]: audit 2024-02-10T08:38:04.568146+0000 mon.c (mon.2) 888 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:05 smithi067 bash[24488]: audit 2024-02-10T08:38:04.568825+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:06.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:05 smithi134 bash[18722]: cluster 2024-02-10T08:38:04.211210+0000 mgr.y (mgr.14547) 607 : cluster [DBG] pgmap v944: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:06.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:05 smithi134 bash[18722]: cluster 2024-02-10T08:38:04.546932+0000 mon.a (mon.0) 2664 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-10T08:38:06.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:05 smithi134 bash[18722]: audit 2024-02-10T08:38:04.568146+0000 mon.c (mon.2) 888 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:06.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:05 smithi134 bash[18722]: audit 2024-02-10T08:38:04.568825+0000 mon.a (mon.0) 2665 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[17385]: audit 2024-02-10T08:38:05.552800+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[17385]: cluster 2024-02-10T08:38:05.553034+0000 mon.a (mon.0) 2667 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[17385]: audit 2024-02-10T08:38:05.556476+0000 mon.c (mon.2) 889 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[17385]: audit 2024-02-10T08:38:05.563174+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[24488]: audit 2024-02-10T08:38:05.552800+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[24488]: cluster 2024-02-10T08:38:05.553034+0000 mon.a (mon.0) 2667 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[24488]: audit 2024-02-10T08:38:05.556476+0000 mon.c (mon.2) 889 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:06 smithi067 bash[24488]: audit 2024-02-10T08:38:05.563174+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:07.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:06 smithi134 bash[18722]: audit 2024-02-10T08:38:05.552800+0000 mon.a (mon.0) 2666 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:07.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:06 smithi134 bash[18722]: cluster 2024-02-10T08:38:05.553034+0000 mon.a (mon.0) 2667 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-10T08:38:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:06 smithi134 bash[18722]: audit 2024-02-10T08:38:05.556476+0000 mon.c (mon.2) 889 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:07.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:06 smithi134 bash[18722]: audit 2024-02-10T08:38:05.563174+0000 mon.a (mon.0) 2668 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:07.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[17385]: cluster 2024-02-10T08:38:06.212025+0000 mgr.y (mgr.14547) 608 : cluster [DBG] pgmap v947: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[17385]: audit 2024-02-10T08:38:06.570985+0000 mon.a (mon.0) 2669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]': finished 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[17385]: cluster 2024-02-10T08:38:06.571642+0000 mon.a (mon.0) 2670 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[17385]: audit 2024-02-10T08:38:06.574067+0000 mon.c (mon.2) 890 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]: dispatch 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[17385]: audit 2024-02-10T08:38:06.582904+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]: dispatch 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[24488]: cluster 2024-02-10T08:38:06.212025+0000 mgr.y (mgr.14547) 608 : cluster [DBG] pgmap v947: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[24488]: audit 2024-02-10T08:38:06.570985+0000 mon.a (mon.0) 2669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]': finished 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[24488]: cluster 2024-02-10T08:38:06.571642+0000 mon.a (mon.0) 2670 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[24488]: audit 2024-02-10T08:38:06.574067+0000 mon.c (mon.2) 890 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]: dispatch 2024-02-10T08:38:07.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[24488]: audit 2024-02-10T08:38:06.582904+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]: dispatch 2024-02-10T08:38:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:07 smithi134 bash[18722]: cluster 2024-02-10T08:38:06.212025+0000 mgr.y (mgr.14547) 608 : cluster [DBG] pgmap v947: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:07 smithi134 bash[18722]: audit 2024-02-10T08:38:06.570985+0000 mon.a (mon.0) 2669 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-106"}]': finished 2024-02-10T08:38:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:07 smithi134 bash[18722]: cluster 2024-02-10T08:38:06.571642+0000 mon.a (mon.0) 2670 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-10T08:38:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:07 smithi134 bash[18722]: audit 2024-02-10T08:38:06.574067+0000 mon.c (mon.2) 890 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]: dispatch 2024-02-10T08:38:08.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:07 smithi134 bash[18722]: audit 2024-02-10T08:38:06.582904+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]: dispatch 2024-02-10T08:38:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:38:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:07] "GET /metrics HTTP/1.1" 200 190304 "" "Prometheus/2.33.4" 2024-02-10T08:38:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[17385]: cluster 2024-02-10T08:38:07.575761+0000 mon.a (mon.0) 2672 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[17385]: audit 2024-02-10T08:38:07.584016+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]': finished 2024-02-10T08:38:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[17385]: cluster 2024-02-10T08:38:07.584185+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-10T08:38:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[17385]: audit 2024-02-10T08:38:07.614637+0000 mon.c (mon.2) 891 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.3"}]: dispatch 2024-02-10T08:38:08.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[17385]: audit 2024-02-10T08:38:07.614937+0000 mgr.y (mgr.14547) 609 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.3"}]: dispatch 2024-02-10T08:38:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[24488]: cluster 2024-02-10T08:38:07.575761+0000 mon.a (mon.0) 2672 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[24488]: audit 2024-02-10T08:38:07.584016+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]': finished 2024-02-10T08:38:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[24488]: cluster 2024-02-10T08:38:07.584185+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-10T08:38:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[24488]: audit 2024-02-10T08:38:07.614637+0000 mon.c (mon.2) 891 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.3"}]: dispatch 2024-02-10T08:38:08.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:08 smithi067 bash[24488]: audit 2024-02-10T08:38:07.614937+0000 mgr.y (mgr.14547) 609 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.3"}]: dispatch 2024-02-10T08:38:09.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:08 smithi134 bash[18722]: cluster 2024-02-10T08:38:07.575761+0000 mon.a (mon.0) 2672 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:09.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:08 smithi134 bash[18722]: audit 2024-02-10T08:38:07.584016+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-106", "mode": "writeback"}]': finished 2024-02-10T08:38:09.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:08 smithi134 bash[18722]: cluster 2024-02-10T08:38:07.584185+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-10T08:38:09.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:08 smithi134 bash[18722]: audit 2024-02-10T08:38:07.614637+0000 mon.c (mon.2) 891 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.3"}]: dispatch 2024-02-10T08:38:09.036 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:08 smithi134 bash[18722]: audit 2024-02-10T08:38:07.614937+0000 mgr.y (mgr.14547) 609 : audit [DBG] from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.3"}]: dispatch 2024-02-10T08:38:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:09 smithi067 bash[17385]: cluster 2024-02-10T08:38:08.212914+0000 mgr.y (mgr.14547) 610 : cluster [DBG] pgmap v950: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:38:09.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:09 smithi067 bash[24488]: cluster 2024-02-10T08:38:08.212914+0000 mgr.y (mgr.14547) 610 : cluster [DBG] pgmap v950: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:38:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:09 smithi134 bash[18722]: cluster 2024-02-10T08:38:08.212914+0000 mgr.y (mgr.14547) 610 : cluster [DBG] pgmap v950: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:38:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:11 smithi067 bash[17385]: cluster 2024-02-10T08:38:10.213742+0000 mgr.y (mgr.14547) 611 : cluster [DBG] pgmap v951: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 903 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:38:11.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:11 smithi067 bash[24488]: cluster 2024-02-10T08:38:10.213742+0000 mgr.y (mgr.14547) 611 : cluster [DBG] pgmap v951: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 903 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:38:12.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:11 smithi134 bash[18722]: cluster 2024-02-10T08:38:10.213742+0000 mgr.y (mgr.14547) 611 : cluster [DBG] pgmap v951: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 903 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:38:12.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:38:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:38:13.276 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:38:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:38:13.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:13 smithi134 bash[18722]: cluster 2024-02-10T08:38:12.215908+0000 mgr.y (mgr.14547) 612 : cluster [DBG] pgmap v952: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-10T08:38:13.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:13 smithi134 bash[18722]: audit 2024-02-10T08:38:12.347063+0000 mgr.y (mgr.14547) 613 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:13 smithi067 bash[17385]: cluster 2024-02-10T08:38:12.215908+0000 mgr.y (mgr.14547) 612 : cluster [DBG] pgmap v952: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-10T08:38:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:13 smithi067 bash[17385]: audit 2024-02-10T08:38:12.347063+0000 mgr.y (mgr.14547) 613 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:13.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:13 smithi067 bash[24488]: cluster 2024-02-10T08:38:12.215908+0000 mgr.y (mgr.14547) 612 : cluster [DBG] pgmap v952: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-10T08:38:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:13 smithi067 bash[24488]: audit 2024-02-10T08:38:12.347063+0000 mgr.y (mgr.14547) 613 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:14.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:14 smithi134 bash[18722]: cluster 2024-02-10T08:38:13.497885+0000 osd.1 (osd.1) 9 : cluster [DBG] 237.3 scrub starts 2024-02-10T08:38:14.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:14 smithi134 bash[18722]: cluster 2024-02-10T08:38:13.500907+0000 osd.1 (osd.1) 10 : cluster [DBG] 237.3 scrub ok 2024-02-10T08:38:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:14 smithi067 bash[17385]: cluster 2024-02-10T08:38:13.497885+0000 osd.1 (osd.1) 9 : cluster [DBG] 237.3 scrub starts 2024-02-10T08:38:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:14 smithi067 bash[17385]: cluster 2024-02-10T08:38:13.500907+0000 osd.1 (osd.1) 10 : cluster [DBG] 237.3 scrub ok 2024-02-10T08:38:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:14 smithi067 bash[24488]: cluster 2024-02-10T08:38:13.497885+0000 osd.1 (osd.1) 9 : cluster [DBG] 237.3 scrub starts 2024-02-10T08:38:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:14 smithi067 bash[24488]: cluster 2024-02-10T08:38:13.500907+0000 osd.1 (osd.1) 10 : cluster [DBG] 237.3 scrub ok 2024-02-10T08:38:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:15 smithi067 bash[17385]: cluster 2024-02-10T08:38:14.216823+0000 mgr.y (mgr.14547) 614 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-10T08:38:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:15 smithi067 bash[24488]: cluster 2024-02-10T08:38:14.216823+0000 mgr.y (mgr.14547) 614 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-10T08:38:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:15 smithi134 bash[18722]: cluster 2024-02-10T08:38:14.216823+0000 mgr.y (mgr.14547) 614 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-02-10T08:38:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:16 smithi067 bash[17385]: audit 2024-02-10T08:38:15.790622+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:17.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:16 smithi067 bash[24488]: audit 2024-02-10T08:38:15.790622+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:17.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:16 smithi134 bash[18722]: audit 2024-02-10T08:38:15.790622+0000 mon.a (mon.0) 2675 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:17 smithi067 bash[17385]: cluster 2024-02-10T08:38:16.217714+0000 mgr.y (mgr.14547) 615 : cluster [DBG] pgmap v954: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 742 B/s wr, 2 op/s 2024-02-10T08:38:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:38:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:17] "GET /metrics HTTP/1.1" 200 190458 "" "Prometheus/2.33.4" 2024-02-10T08:38:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:17 smithi067 bash[24488]: cluster 2024-02-10T08:38:16.217714+0000 mgr.y (mgr.14547) 615 : cluster [DBG] pgmap v954: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 742 B/s wr, 2 op/s 2024-02-10T08:38:18.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:17 smithi134 bash[18722]: cluster 2024-02-10T08:38:16.217714+0000 mgr.y (mgr.14547) 615 : cluster [DBG] pgmap v954: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 742 B/s wr, 2 op/s 2024-02-10T08:38:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:19 smithi067 bash[17385]: cluster 2024-02-10T08:38:18.219774+0000 mgr.y (mgr.14547) 616 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:38:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:19 smithi067 bash[24488]: cluster 2024-02-10T08:38:18.219774+0000 mgr.y (mgr.14547) 616 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:38:20.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:19 smithi134 bash[18722]: cluster 2024-02-10T08:38:18.219774+0000 mgr.y (mgr.14547) 616 : cluster [DBG] pgmap v955: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:38:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:21 smithi067 bash[17385]: cluster 2024-02-10T08:38:20.220588+0000 mgr.y (mgr.14547) 617 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:21 smithi067 bash[24488]: cluster 2024-02-10T08:38:20.220588+0000 mgr.y (mgr.14547) 617 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:22.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:21 smithi134 bash[18722]: cluster 2024-02-10T08:38:20.220588+0000 mgr.y (mgr.14547) 617 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:22.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:38:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:38:23.276 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:38:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:38:23.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:23 smithi134 bash[18722]: cluster 2024-02-10T08:38:22.222718+0000 mgr.y (mgr.14547) 618 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:38:23.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:23 smithi134 bash[18722]: audit 2024-02-10T08:38:22.352755+0000 mgr.y (mgr.14547) 619 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:23.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:23 smithi067 bash[24488]: cluster 2024-02-10T08:38:22.222718+0000 mgr.y (mgr.14547) 618 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:38:23.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:23 smithi067 bash[24488]: audit 2024-02-10T08:38:22.352755+0000 mgr.y (mgr.14547) 619 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:23 smithi067 bash[17385]: cluster 2024-02-10T08:38:22.222718+0000 mgr.y (mgr.14547) 618 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:38:23.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:23 smithi067 bash[17385]: audit 2024-02-10T08:38:22.352755+0000 mgr.y (mgr.14547) 619 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:24 smithi134 bash[18722]: cluster 2024-02-10T08:38:23.021041+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-10T08:38:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:24 smithi134 bash[18722]: audit 2024-02-10T08:38:23.076464+0000 mon.c (mon.2) 892 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:24 smithi134 bash[18722]: audit 2024-02-10T08:38:23.076980+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:24 smithi067 bash[17385]: cluster 2024-02-10T08:38:23.021041+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-10T08:38:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:24 smithi067 bash[17385]: audit 2024-02-10T08:38:23.076464+0000 mon.c (mon.2) 892 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:24 smithi067 bash[17385]: audit 2024-02-10T08:38:23.076980+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:24.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:24 smithi067 bash[24488]: cluster 2024-02-10T08:38:23.021041+0000 mon.a (mon.0) 2676 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-10T08:38:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:24 smithi067 bash[24488]: audit 2024-02-10T08:38:23.076464+0000 mon.c (mon.2) 892 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:24 smithi067 bash[24488]: audit 2024-02-10T08:38:23.076980+0000 mon.a (mon.0) 2677 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:25.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[17385]: audit 2024-02-10T08:38:24.032976+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:25.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[17385]: cluster 2024-02-10T08:38:24.033082+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-10T08:38:25.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[17385]: audit 2024-02-10T08:38:24.035894+0000 mon.c (mon.2) 893 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:25.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[17385]: audit 2024-02-10T08:38:24.036450+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:25.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[17385]: cluster 2024-02-10T08:38:24.223529+0000 mgr.y (mgr.14547) 620 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:25.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[24488]: audit 2024-02-10T08:38:24.032976+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:25.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[24488]: cluster 2024-02-10T08:38:24.033082+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-10T08:38:25.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[24488]: audit 2024-02-10T08:38:24.035894+0000 mon.c (mon.2) 893 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:25.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[24488]: audit 2024-02-10T08:38:24.036450+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:25.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:25 smithi067 bash[24488]: cluster 2024-02-10T08:38:24.223529+0000 mgr.y (mgr.14547) 620 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:25 smithi134 bash[18722]: audit 2024-02-10T08:38:24.032976+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:25 smithi134 bash[18722]: cluster 2024-02-10T08:38:24.033082+0000 mon.a (mon.0) 2679 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-10T08:38:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:25 smithi134 bash[18722]: audit 2024-02-10T08:38:24.035894+0000 mon.c (mon.2) 893 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:25 smithi134 bash[18722]: audit 2024-02-10T08:38:24.036450+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]: dispatch 2024-02-10T08:38:25.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:25 smithi134 bash[18722]: cluster 2024-02-10T08:38:24.223529+0000 mgr.y (mgr.14547) 620 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:26 smithi067 bash[17385]: cluster 2024-02-10T08:38:25.029746+0000 mon.a (mon.0) 2681 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:26 smithi067 bash[17385]: audit 2024-02-10T08:38:25.038978+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]': finished 2024-02-10T08:38:26.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:26 smithi067 bash[17385]: cluster 2024-02-10T08:38:25.039087+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-10T08:38:26.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:26 smithi067 bash[24488]: cluster 2024-02-10T08:38:25.029746+0000 mon.a (mon.0) 2681 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:26 smithi067 bash[24488]: audit 2024-02-10T08:38:25.038978+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]': finished 2024-02-10T08:38:26.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:26 smithi067 bash[24488]: cluster 2024-02-10T08:38:25.039087+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-10T08:38:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:26 smithi134 bash[18722]: cluster 2024-02-10T08:38:25.029746+0000 mon.a (mon.0) 2681 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:26 smithi134 bash[18722]: audit 2024-02-10T08:38:25.038978+0000 mon.a (mon.0) 2682 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-106"}]': finished 2024-02-10T08:38:26.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:26 smithi134 bash[18722]: cluster 2024-02-10T08:38:25.039087+0000 mon.a (mon.0) 2683 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-10T08:38:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:27 smithi067 bash[17385]: cluster 2024-02-10T08:38:26.057133+0000 mon.a (mon.0) 2684 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-10T08:38:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:27 smithi067 bash[17385]: cluster 2024-02-10T08:38:26.224294+0000 mgr.y (mgr.14547) 621 : cluster [DBG] pgmap v963: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:27.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:27 smithi067 bash[24488]: cluster 2024-02-10T08:38:26.057133+0000 mon.a (mon.0) 2684 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-10T08:38:27.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:27 smithi067 bash[24488]: cluster 2024-02-10T08:38:26.224294+0000 mgr.y (mgr.14547) 621 : cluster [DBG] pgmap v963: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:27.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:27 smithi134 bash[18722]: cluster 2024-02-10T08:38:26.057133+0000 mon.a (mon.0) 2684 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-10T08:38:27.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:27 smithi134 bash[18722]: cluster 2024-02-10T08:38:26.224294+0000 mgr.y (mgr.14547) 621 : cluster [DBG] pgmap v963: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:38:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:27] "GET /metrics HTTP/1.1" 200 190458 "" "Prometheus/2.33.4" 2024-02-10T08:38:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:28 smithi067 bash[17385]: cluster 2024-02-10T08:38:27.067083+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-10T08:38:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:28 smithi067 bash[17385]: audit 2024-02-10T08:38:27.069617+0000 mon.c (mon.2) 894 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:28 smithi067 bash[17385]: audit 2024-02-10T08:38:27.070876+0000 mon.a (mon.0) 2686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:28 smithi067 bash[24488]: cluster 2024-02-10T08:38:27.067083+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-10T08:38:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:28 smithi067 bash[24488]: audit 2024-02-10T08:38:27.069617+0000 mon.c (mon.2) 894 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:28 smithi067 bash[24488]: audit 2024-02-10T08:38:27.070876+0000 mon.a (mon.0) 2686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:28.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:28 smithi134 bash[18722]: cluster 2024-02-10T08:38:27.067083+0000 mon.a (mon.0) 2685 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-10T08:38:28.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:28 smithi134 bash[18722]: audit 2024-02-10T08:38:27.069617+0000 mon.c (mon.2) 894 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:28.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:28 smithi134 bash[18722]: audit 2024-02-10T08:38:27.070876+0000 mon.a (mon.0) 2686 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:29 smithi067 bash[17385]: audit 2024-02-10T08:38:28.077826+0000 mon.a (mon.0) 2687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:29 smithi067 bash[17385]: cluster 2024-02-10T08:38:28.078000+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-10T08:38:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:29 smithi067 bash[17385]: cluster 2024-02-10T08:38:28.225139+0000 mgr.y (mgr.14547) 622 : cluster [DBG] pgmap v966: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T08:38:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:29 smithi067 bash[24488]: audit 2024-02-10T08:38:28.077826+0000 mon.a (mon.0) 2687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:29 smithi067 bash[24488]: cluster 2024-02-10T08:38:28.078000+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-10T08:38:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:29 smithi067 bash[24488]: cluster 2024-02-10T08:38:28.225139+0000 mgr.y (mgr.14547) 622 : cluster [DBG] pgmap v966: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T08:38:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:29 smithi134 bash[18722]: audit 2024-02-10T08:38:28.077826+0000 mon.a (mon.0) 2687 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:29 smithi134 bash[18722]: cluster 2024-02-10T08:38:28.078000+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-10T08:38:29.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:29 smithi134 bash[18722]: cluster 2024-02-10T08:38:28.225139+0000 mgr.y (mgr.14547) 622 : cluster [DBG] pgmap v966: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-10T08:38:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:30 smithi067 bash[17385]: cluster 2024-02-10T08:38:29.087842+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-10T08:38:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:30 smithi067 bash[17385]: audit 2024-02-10T08:38:29.097574+0000 mon.c (mon.2) 895 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:30 smithi067 bash[17385]: audit 2024-02-10T08:38:29.098596+0000 mon.a (mon.0) 2690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:30 smithi067 bash[24488]: cluster 2024-02-10T08:38:29.087842+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-10T08:38:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:30 smithi067 bash[24488]: audit 2024-02-10T08:38:29.097574+0000 mon.c (mon.2) 895 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:30 smithi067 bash[24488]: audit 2024-02-10T08:38:29.098596+0000 mon.a (mon.0) 2690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:30.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:30 smithi134 bash[18722]: cluster 2024-02-10T08:38:29.087842+0000 mon.a (mon.0) 2689 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-10T08:38:30.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:30 smithi134 bash[18722]: audit 2024-02-10T08:38:29.097574+0000 mon.c (mon.2) 895 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:30.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:30 smithi134 bash[18722]: audit 2024-02-10T08:38:29.098596+0000 mon.a (mon.0) 2690 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[17385]: audit 2024-02-10T08:38:30.092028+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[17385]: cluster 2024-02-10T08:38:30.092169+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-10T08:38:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[17385]: audit 2024-02-10T08:38:30.093612+0000 mon.c (mon.2) 896 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[17385]: audit 2024-02-10T08:38:30.099952+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[17385]: cluster 2024-02-10T08:38:30.225762+0000 mgr.y (mgr.14547) 623 : cluster [DBG] pgmap v969: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[17385]: audit 2024-02-10T08:38:30.790558+0000 mon.a (mon.0) 2694 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[24488]: audit 2024-02-10T08:38:30.092028+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[24488]: cluster 2024-02-10T08:38:30.092169+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[24488]: audit 2024-02-10T08:38:30.093612+0000 mon.c (mon.2) 896 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[24488]: audit 2024-02-10T08:38:30.099952+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[24488]: cluster 2024-02-10T08:38:30.225762+0000 mgr.y (mgr.14547) 623 : cluster [DBG] pgmap v969: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:31 smithi067 bash[24488]: audit 2024-02-10T08:38:30.790558+0000 mon.a (mon.0) 2694 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:31.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:31 smithi134 bash[18722]: audit 2024-02-10T08:38:30.092028+0000 mon.a (mon.0) 2691 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:31.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:31 smithi134 bash[18722]: cluster 2024-02-10T08:38:30.092169+0000 mon.a (mon.0) 2692 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-10T08:38:31.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:31 smithi134 bash[18722]: audit 2024-02-10T08:38:30.093612+0000 mon.c (mon.2) 896 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:31 smithi134 bash[18722]: audit 2024-02-10T08:38:30.099952+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:31 smithi134 bash[18722]: cluster 2024-02-10T08:38:30.225762+0000 mgr.y (mgr.14547) 623 : cluster [DBG] pgmap v969: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:38:31.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:31 smithi134 bash[18722]: audit 2024-02-10T08:38:30.790558+0000 mon.a (mon.0) 2694 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:32.365 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: audit 2024-02-10T08:38:31.104990+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]': finished 2024-02-10T08:38:32.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: cluster 2024-02-10T08:38:31.105229+0000 mon.a (mon.0) 2696 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-10T08:38:32.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: audit 2024-02-10T08:38:31.107751+0000 mon.c (mon.2) 897 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]: dispatch 2024-02-10T08:38:32.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: audit 2024-02-10T08:38:31.113610+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]: dispatch 2024-02-10T08:38:32.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: cluster 2024-02-10T08:38:32.009864+0000 mon.a (mon.0) 2698 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:32.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: audit 2024-02-10T08:38:32.018807+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]': finished 2024-02-10T08:38:32.366 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[18722]: cluster 2024-02-10T08:38:32.018948+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-10T08:38:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: audit 2024-02-10T08:38:31.104990+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]': finished 2024-02-10T08:38:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: cluster 2024-02-10T08:38:31.105229+0000 mon.a (mon.0) 2696 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-10T08:38:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: audit 2024-02-10T08:38:31.107751+0000 mon.c (mon.2) 897 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]: dispatch 2024-02-10T08:38:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: audit 2024-02-10T08:38:31.113610+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]: dispatch 2024-02-10T08:38:32.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: cluster 2024-02-10T08:38:32.009864+0000 mon.a (mon.0) 2698 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: audit 2024-02-10T08:38:32.018807+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]': finished 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[24488]: cluster 2024-02-10T08:38:32.018948+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: audit 2024-02-10T08:38:31.104990+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-108"}]': finished 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: cluster 2024-02-10T08:38:31.105229+0000 mon.a (mon.0) 2696 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: audit 2024-02-10T08:38:31.107751+0000 mon.c (mon.2) 897 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]: dispatch 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: audit 2024-02-10T08:38:31.113610+0000 mon.a (mon.0) 2697 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]: dispatch 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: cluster 2024-02-10T08:38:32.009864+0000 mon.a (mon.0) 2698 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: audit 2024-02-10T08:38:32.018807+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-108", "mode": "writeback"}]': finished 2024-02-10T08:38:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:32 smithi067 bash[17385]: cluster 2024-02-10T08:38:32.018948+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-10T08:38:32.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:38:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:38:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:38:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:38:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:33 smithi134 bash[18722]: cluster 2024-02-10T08:38:32.226570+0000 mgr.y (mgr.14547) 624 : cluster [DBG] pgmap v972: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:33 smithi134 bash[18722]: audit 2024-02-10T08:38:32.364263+0000 mgr.y (mgr.14547) 625 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:33 smithi134 bash[18722]: cluster 2024-02-10T08:38:33.032386+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-10T08:38:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:33 smithi134 bash[18722]: audit 2024-02-10T08:38:33.082371+0000 mon.c (mon.2) 898 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:33.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:33 smithi134 bash[18722]: audit 2024-02-10T08:38:33.082908+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[17385]: cluster 2024-02-10T08:38:32.226570+0000 mgr.y (mgr.14547) 624 : cluster [DBG] pgmap v972: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[17385]: audit 2024-02-10T08:38:32.364263+0000 mgr.y (mgr.14547) 625 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[17385]: cluster 2024-02-10T08:38:33.032386+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-10T08:38:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[17385]: audit 2024-02-10T08:38:33.082371+0000 mon.c (mon.2) 898 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:33.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[17385]: audit 2024-02-10T08:38:33.082908+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[24488]: cluster 2024-02-10T08:38:32.226570+0000 mgr.y (mgr.14547) 624 : cluster [DBG] pgmap v972: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[24488]: audit 2024-02-10T08:38:32.364263+0000 mgr.y (mgr.14547) 625 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[24488]: cluster 2024-02-10T08:38:33.032386+0000 mon.a (mon.0) 2701 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-10T08:38:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[24488]: audit 2024-02-10T08:38:33.082371+0000 mon.c (mon.2) 898 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:33.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:33 smithi067 bash[24488]: audit 2024-02-10T08:38:33.082908+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[17385]: audit 2024-02-10T08:38:34.033980+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[17385]: cluster 2024-02-10T08:38:34.034141+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-10T08:38:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[17385]: audit 2024-02-10T08:38:34.035935+0000 mon.c (mon.2) 899 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[17385]: audit 2024-02-10T08:38:34.036459+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[17385]: cluster 2024-02-10T08:38:34.227336+0000 mgr.y (mgr.14547) 626 : cluster [DBG] pgmap v975: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[24488]: audit 2024-02-10T08:38:34.033980+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[24488]: cluster 2024-02-10T08:38:34.034141+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[24488]: audit 2024-02-10T08:38:34.035935+0000 mon.c (mon.2) 899 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[24488]: audit 2024-02-10T08:38:34.036459+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:35 smithi067 bash[24488]: cluster 2024-02-10T08:38:34.227336+0000 mgr.y (mgr.14547) 626 : cluster [DBG] pgmap v975: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:35.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:35 smithi134 bash[18722]: audit 2024-02-10T08:38:34.033980+0000 mon.a (mon.0) 2703 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:35 smithi134 bash[18722]: cluster 2024-02-10T08:38:34.034141+0000 mon.a (mon.0) 2704 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-10T08:38:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:35 smithi134 bash[18722]: audit 2024-02-10T08:38:34.035935+0000 mon.c (mon.2) 899 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:35 smithi134 bash[18722]: audit 2024-02-10T08:38:34.036459+0000 mon.a (mon.0) 2705 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]: dispatch 2024-02-10T08:38:35.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:35 smithi134 bash[18722]: cluster 2024-02-10T08:38:34.227336+0000 mgr.y (mgr.14547) 626 : cluster [DBG] pgmap v975: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:36.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:36 smithi067 bash[24488]: cluster 2024-02-10T08:38:35.028562+0000 mon.a (mon.0) 2706 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:36.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:36 smithi067 bash[24488]: audit 2024-02-10T08:38:35.040774+0000 mon.a (mon.0) 2707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]': finished 2024-02-10T08:38:36.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:36 smithi067 bash[24488]: cluster 2024-02-10T08:38:35.041052+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-10T08:38:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:36 smithi067 bash[17385]: cluster 2024-02-10T08:38:35.028562+0000 mon.a (mon.0) 2706 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:36.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:36 smithi067 bash[17385]: audit 2024-02-10T08:38:35.040774+0000 mon.a (mon.0) 2707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]': finished 2024-02-10T08:38:36.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:36 smithi067 bash[17385]: cluster 2024-02-10T08:38:35.041052+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-10T08:38:36.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:36 smithi134 bash[18722]: cluster 2024-02-10T08:38:35.028562+0000 mon.a (mon.0) 2706 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:36.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:36 smithi134 bash[18722]: audit 2024-02-10T08:38:35.040774+0000 mon.a (mon.0) 2707 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-108"}]': finished 2024-02-10T08:38:36.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:36 smithi134 bash[18722]: cluster 2024-02-10T08:38:35.041052+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-10T08:38:37.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:37 smithi067 bash[17385]: cluster 2024-02-10T08:38:36.056365+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-10T08:38:37.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:37 smithi067 bash[17385]: cluster 2024-02-10T08:38:36.228013+0000 mgr.y (mgr.14547) 627 : cluster [DBG] pgmap v978: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:37 smithi067 bash[24488]: cluster 2024-02-10T08:38:36.056365+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-10T08:38:37.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:37 smithi067 bash[24488]: cluster 2024-02-10T08:38:36.228013+0000 mgr.y (mgr.14547) 627 : cluster [DBG] pgmap v978: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:37.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:37 smithi134 bash[18722]: cluster 2024-02-10T08:38:36.056365+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-10T08:38:37.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:37 smithi134 bash[18722]: cluster 2024-02-10T08:38:36.228013+0000 mgr.y (mgr.14547) 627 : cluster [DBG] pgmap v978: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:38:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:37] "GET /metrics HTTP/1.1" 200 190539 "" "Prometheus/2.33.4" 2024-02-10T08:38:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:38 smithi067 bash[24488]: cluster 2024-02-10T08:38:37.069784+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-10T08:38:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:38 smithi067 bash[24488]: audit 2024-02-10T08:38:37.077954+0000 mon.c (mon.2) 900 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:38.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:38 smithi067 bash[24488]: audit 2024-02-10T08:38:37.079460+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:38 smithi067 bash[17385]: cluster 2024-02-10T08:38:37.069784+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-10T08:38:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:38 smithi067 bash[17385]: audit 2024-02-10T08:38:37.077954+0000 mon.c (mon.2) 900 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:38.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:38 smithi067 bash[17385]: audit 2024-02-10T08:38:37.079460+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:38.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:38 smithi134 bash[18722]: cluster 2024-02-10T08:38:37.069784+0000 mon.a (mon.0) 2710 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-10T08:38:38.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:38 smithi134 bash[18722]: audit 2024-02-10T08:38:37.077954+0000 mon.c (mon.2) 900 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:38.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:38 smithi134 bash[18722]: audit 2024-02-10T08:38:37.079460+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[17385]: audit 2024-02-10T08:38:38.075483+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[17385]: cluster 2024-02-10T08:38:38.075725+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-10T08:38:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[17385]: audit 2024-02-10T08:38:38.100863+0000 mon.c (mon.2) 901 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[17385]: audit 2024-02-10T08:38:38.102058+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[17385]: cluster 2024-02-10T08:38:38.228868+0000 mgr.y (mgr.14547) 628 : cluster [DBG] pgmap v981: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[24488]: audit 2024-02-10T08:38:38.075483+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[24488]: cluster 2024-02-10T08:38:38.075725+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[24488]: audit 2024-02-10T08:38:38.100863+0000 mon.c (mon.2) 901 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[24488]: audit 2024-02-10T08:38:38.102058+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:39.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:39 smithi067 bash[24488]: cluster 2024-02-10T08:38:38.228868+0000 mgr.y (mgr.14547) 628 : cluster [DBG] pgmap v981: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:39.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:39 smithi134 bash[18722]: audit 2024-02-10T08:38:38.075483+0000 mon.a (mon.0) 2712 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:39.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:39 smithi134 bash[18722]: cluster 2024-02-10T08:38:38.075725+0000 mon.a (mon.0) 2713 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-10T08:38:39.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:39 smithi134 bash[18722]: audit 2024-02-10T08:38:38.100863+0000 mon.c (mon.2) 901 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:39 smithi134 bash[18722]: audit 2024-02-10T08:38:38.102058+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:39.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:39 smithi134 bash[18722]: cluster 2024-02-10T08:38:38.228868+0000 mgr.y (mgr.14547) 628 : cluster [DBG] pgmap v981: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[17385]: audit 2024-02-10T08:38:39.088560+0000 mon.a (mon.0) 2715 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[17385]: cluster 2024-02-10T08:38:39.088692+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-10T08:38:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[17385]: audit 2024-02-10T08:38:39.091053+0000 mon.c (mon.2) 902 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:40.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[17385]: audit 2024-02-10T08:38:39.098661+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[24488]: audit 2024-02-10T08:38:39.088560+0000 mon.a (mon.0) 2715 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[24488]: cluster 2024-02-10T08:38:39.088692+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-10T08:38:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[24488]: audit 2024-02-10T08:38:39.091053+0000 mon.c (mon.2) 902 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:40 smithi067 bash[24488]: audit 2024-02-10T08:38:39.098661+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:40.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:40 smithi134 bash[18722]: audit 2024-02-10T08:38:39.088560+0000 mon.a (mon.0) 2715 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:40.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:40 smithi134 bash[18722]: cluster 2024-02-10T08:38:39.088692+0000 mon.a (mon.0) 2716 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-10T08:38:40.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:40 smithi134 bash[18722]: audit 2024-02-10T08:38:39.091053+0000 mon.c (mon.2) 902 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:40.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:40 smithi134 bash[18722]: audit 2024-02-10T08:38:39.098661+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:41.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:41 smithi134 bash[18722]: audit 2024-02-10T08:38:40.101686+0000 mon.a (mon.0) 2718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]': finished 2024-02-10T08:38:41.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:41 smithi134 bash[18722]: cluster 2024-02-10T08:38:40.101820+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-10T08:38:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:41 smithi134 bash[18722]: audit 2024-02-10T08:38:40.104733+0000 mon.c (mon.2) 903 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]: dispatch 2024-02-10T08:38:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:41 smithi134 bash[18722]: audit 2024-02-10T08:38:40.112034+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]: dispatch 2024-02-10T08:38:41.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:41 smithi134 bash[18722]: cluster 2024-02-10T08:38:40.229750+0000 mgr.y (mgr.14547) 629 : cluster [DBG] pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[24488]: audit 2024-02-10T08:38:40.101686+0000 mon.a (mon.0) 2718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]': finished 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[24488]: cluster 2024-02-10T08:38:40.101820+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[24488]: audit 2024-02-10T08:38:40.104733+0000 mon.c (mon.2) 903 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]: dispatch 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[24488]: audit 2024-02-10T08:38:40.112034+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]: dispatch 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[24488]: cluster 2024-02-10T08:38:40.229750+0000 mgr.y (mgr.14547) 629 : cluster [DBG] pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[17385]: audit 2024-02-10T08:38:40.101686+0000 mon.a (mon.0) 2718 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-110"}]': finished 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[17385]: cluster 2024-02-10T08:38:40.101820+0000 mon.a (mon.0) 2719 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[17385]: audit 2024-02-10T08:38:40.104733+0000 mon.c (mon.2) 903 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]: dispatch 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[17385]: audit 2024-02-10T08:38:40.112034+0000 mon.a (mon.0) 2720 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]: dispatch 2024-02-10T08:38:41.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:41 smithi067 bash[17385]: cluster 2024-02-10T08:38:40.229750+0000 mgr.y (mgr.14547) 629 : cluster [DBG] pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-10T08:38:42.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:38:42.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: cluster 2024-02-10T08:38:41.108073+0000 mon.a (mon.0) 2721 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:42.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: audit 2024-02-10T08:38:41.117038+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]': finished 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: cluster 2024-02-10T08:38:41.117163+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: audit 2024-02-10T08:38:41.230190+0000 mon.c (mon.2) 904 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: audit 2024-02-10T08:38:41.230794+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: audit 2024-02-10T08:38:42.026428+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: cluster 2024-02-10T08:38:42.026707+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: audit 2024-02-10T08:38:42.029009+0000 mon.c (mon.2) 905 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:42.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:42 smithi134 bash[18722]: audit 2024-02-10T08:38:42.030435+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:42.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: cluster 2024-02-10T08:38:41.108073+0000 mon.a (mon.0) 2721 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:42.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: audit 2024-02-10T08:38:41.117038+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]': finished 2024-02-10T08:38:42.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: cluster 2024-02-10T08:38:41.117163+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-10T08:38:42.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: audit 2024-02-10T08:38:41.230190+0000 mon.c (mon.2) 904 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: audit 2024-02-10T08:38:41.230794+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: audit 2024-02-10T08:38:42.026428+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: cluster 2024-02-10T08:38:42.026707+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: audit 2024-02-10T08:38:42.029009+0000 mon.c (mon.2) 905 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[17385]: audit 2024-02-10T08:38:42.030435+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: cluster 2024-02-10T08:38:41.108073+0000 mon.a (mon.0) 2721 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: audit 2024-02-10T08:38:41.117038+0000 mon.a (mon.0) 2722 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-110", "mode": "writeback"}]': finished 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: cluster 2024-02-10T08:38:41.117163+0000 mon.a (mon.0) 2723 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: audit 2024-02-10T08:38:41.230190+0000 mon.c (mon.2) 904 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: audit 2024-02-10T08:38:41.230794+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: audit 2024-02-10T08:38:42.026428+0000 mon.a (mon.0) 2725 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: cluster 2024-02-10T08:38:42.026707+0000 mon.a (mon.0) 2726 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: audit 2024-02-10T08:38:42.029009+0000 mon.c (mon.2) 905 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:42.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:42 smithi067 bash[24488]: audit 2024-02-10T08:38:42.030435+0000 mon.a (mon.0) 2727 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]: dispatch 2024-02-10T08:38:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:38:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:38:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:43 smithi134 bash[18722]: cluster 2024-02-10T08:38:42.230591+0000 mgr.y (mgr.14547) 630 : cluster [DBG] pgmap v987: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T08:38:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:43 smithi134 bash[18722]: audit 2024-02-10T08:38:42.375842+0000 mgr.y (mgr.14547) 631 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:43 smithi134 bash[18722]: cluster 2024-02-10T08:38:43.017506+0000 mon.a (mon.0) 2728 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:43 smithi134 bash[18722]: audit 2024-02-10T08:38:43.025524+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]': finished 2024-02-10T08:38:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:43 smithi134 bash[18722]: cluster 2024-02-10T08:38:43.025648+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-10T08:38:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[24488]: cluster 2024-02-10T08:38:42.230591+0000 mgr.y (mgr.14547) 630 : cluster [DBG] pgmap v987: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T08:38:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[24488]: audit 2024-02-10T08:38:42.375842+0000 mgr.y (mgr.14547) 631 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[24488]: cluster 2024-02-10T08:38:43.017506+0000 mon.a (mon.0) 2728 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[24488]: audit 2024-02-10T08:38:43.025524+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]': finished 2024-02-10T08:38:43.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[24488]: cluster 2024-02-10T08:38:43.025648+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-10T08:38:43.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[17385]: cluster 2024-02-10T08:38:42.230591+0000 mgr.y (mgr.14547) 630 : cluster [DBG] pgmap v987: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-10T08:38:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[17385]: audit 2024-02-10T08:38:42.375842+0000 mgr.y (mgr.14547) 631 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[17385]: cluster 2024-02-10T08:38:43.017506+0000 mon.a (mon.0) 2728 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[17385]: audit 2024-02-10T08:38:43.025524+0000 mon.a (mon.0) 2729 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-110"}]': finished 2024-02-10T08:38:43.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:43 smithi067 bash[17385]: cluster 2024-02-10T08:38:43.025648+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-10T08:38:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:45 smithi067 bash[17385]: cluster 2024-02-10T08:38:44.029562+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-10T08:38:45.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:45 smithi067 bash[17385]: cluster 2024-02-10T08:38:44.231367+0000 mgr.y (mgr.14547) 632 : cluster [DBG] pgmap v990: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:45 smithi067 bash[24488]: cluster 2024-02-10T08:38:44.029562+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-10T08:38:45.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:45 smithi067 bash[24488]: cluster 2024-02-10T08:38:44.231367+0000 mgr.y (mgr.14547) 632 : cluster [DBG] pgmap v990: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:45 smithi134 bash[18722]: cluster 2024-02-10T08:38:44.029562+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-10T08:38:45.528 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:45 smithi134 bash[18722]: cluster 2024-02-10T08:38:44.231367+0000 mgr.y (mgr.14547) 632 : cluster [DBG] pgmap v990: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:38:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:46 smithi067 bash[24488]: cluster 2024-02-10T08:38:45.038339+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-10T08:38:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:46 smithi067 bash[24488]: audit 2024-02-10T08:38:45.041121+0000 mon.c (mon.2) 906 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:46 smithi067 bash[24488]: audit 2024-02-10T08:38:45.041852+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:46 smithi067 bash[17385]: cluster 2024-02-10T08:38:45.038339+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-10T08:38:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:46 smithi067 bash[17385]: audit 2024-02-10T08:38:45.041121+0000 mon.c (mon.2) 906 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:46 smithi067 bash[17385]: audit 2024-02-10T08:38:45.041852+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:46.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:46 smithi134 bash[18722]: cluster 2024-02-10T08:38:45.038339+0000 mon.a (mon.0) 2732 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-10T08:38:46.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:46 smithi134 bash[18722]: audit 2024-02-10T08:38:45.041121+0000 mon.c (mon.2) 906 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:46.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:46 smithi134 bash[18722]: audit 2024-02-10T08:38:45.041852+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[17385]: audit 2024-02-10T08:38:46.043112+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[17385]: cluster 2024-02-10T08:38:46.043447+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-10T08:38:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[17385]: audit 2024-02-10T08:38:46.065150+0000 mon.c (mon.2) 907 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[17385]: audit 2024-02-10T08:38:46.065667+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[17385]: cluster 2024-02-10T08:38:46.232203+0000 mgr.y (mgr.14547) 633 : cluster [DBG] pgmap v993: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[24488]: audit 2024-02-10T08:38:46.043112+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[24488]: cluster 2024-02-10T08:38:46.043447+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-10T08:38:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[24488]: audit 2024-02-10T08:38:46.065150+0000 mon.c (mon.2) 907 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[24488]: audit 2024-02-10T08:38:46.065667+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[24488]: cluster 2024-02-10T08:38:46.232203+0000 mgr.y (mgr.14547) 633 : cluster [DBG] pgmap v993: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:47.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:47 smithi134 bash[18722]: audit 2024-02-10T08:38:46.043112+0000 mon.a (mon.0) 2734 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:47.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:47 smithi134 bash[18722]: cluster 2024-02-10T08:38:46.043447+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-10T08:38:47.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:47 smithi134 bash[18722]: audit 2024-02-10T08:38:46.065150+0000 mon.c (mon.2) 907 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:47.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:47 smithi134 bash[18722]: audit 2024-02-10T08:38:46.065667+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:47.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:47 smithi134 bash[18722]: cluster 2024-02-10T08:38:46.232203+0000 mgr.y (mgr.14547) 633 : cluster [DBG] pgmap v993: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:38:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:47] "GET /metrics HTTP/1.1" 200 190662 "" "Prometheus/2.33.4" 2024-02-10T08:38:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[17385]: audit 2024-02-10T08:38:47.059499+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[17385]: cluster 2024-02-10T08:38:47.059686+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[17385]: audit 2024-02-10T08:38:47.063045+0000 mon.c (mon.2) 908 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[17385]: audit 2024-02-10T08:38:47.070448+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[24488]: audit 2024-02-10T08:38:47.059499+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[24488]: cluster 2024-02-10T08:38:47.059686+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[24488]: audit 2024-02-10T08:38:47.063045+0000 mon.c (mon.2) 908 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:48.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:48 smithi067 bash[24488]: audit 2024-02-10T08:38:47.070448+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:48.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:48 smithi134 bash[18722]: audit 2024-02-10T08:38:47.059499+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:48.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:48 smithi134 bash[18722]: cluster 2024-02-10T08:38:47.059686+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-10T08:38:48.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:48 smithi134 bash[18722]: audit 2024-02-10T08:38:47.063045+0000 mon.c (mon.2) 908 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:48.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:48 smithi134 bash[18722]: audit 2024-02-10T08:38:47.070448+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[17385]: audit 2024-02-10T08:38:48.071655+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]': finished 2024-02-10T08:38:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[17385]: cluster 2024-02-10T08:38:48.071780+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-10T08:38:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[17385]: audit 2024-02-10T08:38:48.075366+0000 mon.c (mon.2) 909 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]: dispatch 2024-02-10T08:38:49.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[17385]: audit 2024-02-10T08:38:48.081938+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]: dispatch 2024-02-10T08:38:49.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[17385]: cluster 2024-02-10T08:38:48.233031+0000 mgr.y (mgr.14547) 634 : cluster [DBG] pgmap v996: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[24488]: audit 2024-02-10T08:38:48.071655+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]': finished 2024-02-10T08:38:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[24488]: cluster 2024-02-10T08:38:48.071780+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-10T08:38:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[24488]: audit 2024-02-10T08:38:48.075366+0000 mon.c (mon.2) 909 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]: dispatch 2024-02-10T08:38:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[24488]: audit 2024-02-10T08:38:48.081938+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]: dispatch 2024-02-10T08:38:49.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:49 smithi067 bash[24488]: cluster 2024-02-10T08:38:48.233031+0000 mgr.y (mgr.14547) 634 : cluster [DBG] pgmap v996: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:49.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:49 smithi134 bash[18722]: audit 2024-02-10T08:38:48.071655+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-112"}]': finished 2024-02-10T08:38:49.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:49 smithi134 bash[18722]: cluster 2024-02-10T08:38:48.071780+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-10T08:38:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:49 smithi134 bash[18722]: audit 2024-02-10T08:38:48.075366+0000 mon.c (mon.2) 909 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]: dispatch 2024-02-10T08:38:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:49 smithi134 bash[18722]: audit 2024-02-10T08:38:48.081938+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]: dispatch 2024-02-10T08:38:49.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:49 smithi134 bash[18722]: cluster 2024-02-10T08:38:48.233031+0000 mgr.y (mgr.14547) 634 : cluster [DBG] pgmap v996: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[24488]: cluster 2024-02-10T08:38:49.077719+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[24488]: audit 2024-02-10T08:38:49.088252+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]': finished 2024-02-10T08:38:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[24488]: cluster 2024-02-10T08:38:49.088354+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-10T08:38:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[24488]: audit 2024-02-10T08:38:49.183370+0000 mon.c (mon.2) 910 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:50.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[24488]: audit 2024-02-10T08:38:49.183999+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[17385]: cluster 2024-02-10T08:38:49.077719+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[17385]: audit 2024-02-10T08:38:49.088252+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]': finished 2024-02-10T08:38:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[17385]: cluster 2024-02-10T08:38:49.088354+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-10T08:38:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[17385]: audit 2024-02-10T08:38:49.183370+0000 mon.c (mon.2) 910 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:50.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:50 smithi067 bash[17385]: audit 2024-02-10T08:38:49.183999+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:50.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:50 smithi134 bash[18722]: cluster 2024-02-10T08:38:49.077719+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:50.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:50 smithi134 bash[18722]: audit 2024-02-10T08:38:49.088252+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-112", "mode": "writeback"}]': finished 2024-02-10T08:38:50.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:50 smithi134 bash[18722]: cluster 2024-02-10T08:38:49.088354+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-10T08:38:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:50 smithi134 bash[18722]: audit 2024-02-10T08:38:49.183370+0000 mon.c (mon.2) 910 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:50.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:50 smithi134 bash[18722]: audit 2024-02-10T08:38:49.183999+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:38:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[24488]: audit 2024-02-10T08:38:50.108123+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[24488]: cluster 2024-02-10T08:38:50.108206+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-10T08:38:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[24488]: audit 2024-02-10T08:38:50.110094+0000 mon.c (mon.2) 911 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[24488]: audit 2024-02-10T08:38:50.114446+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:51.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[24488]: cluster 2024-02-10T08:38:50.237319+0000 mgr.y (mgr.14547) 635 : cluster [DBG] pgmap v999: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[17385]: audit 2024-02-10T08:38:50.108123+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[17385]: cluster 2024-02-10T08:38:50.108206+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-10T08:38:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[17385]: audit 2024-02-10T08:38:50.110094+0000 mon.c (mon.2) 911 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[17385]: audit 2024-02-10T08:38:50.114446+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:51.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:51 smithi067 bash[17385]: cluster 2024-02-10T08:38:50.237319+0000 mgr.y (mgr.14547) 635 : cluster [DBG] pgmap v999: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:51.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:51 smithi134 bash[18722]: audit 2024-02-10T08:38:50.108123+0000 mon.a (mon.0) 2747 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:38:51.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:51 smithi134 bash[18722]: cluster 2024-02-10T08:38:50.108206+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-10T08:38:51.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:51 smithi134 bash[18722]: audit 2024-02-10T08:38:50.110094+0000 mon.c (mon.2) 911 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:51.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:51 smithi134 bash[18722]: audit 2024-02-10T08:38:50.114446+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]: dispatch 2024-02-10T08:38:51.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:51 smithi134 bash[18722]: cluster 2024-02-10T08:38:50.237319+0000 mgr.y (mgr.14547) 635 : cluster [DBG] pgmap v999: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:52 smithi067 bash[17385]: cluster 2024-02-10T08:38:51.106287+0000 mon.a (mon.0) 2750 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:52 smithi067 bash[17385]: audit 2024-02-10T08:38:51.122536+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]': finished 2024-02-10T08:38:52.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:52 smithi067 bash[17385]: cluster 2024-02-10T08:38:51.122632+0000 mon.a (mon.0) 2752 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-10T08:38:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:52 smithi067 bash[24488]: cluster 2024-02-10T08:38:51.106287+0000 mon.a (mon.0) 2750 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:52.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:52 smithi067 bash[24488]: audit 2024-02-10T08:38:51.122536+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]': finished 2024-02-10T08:38:52.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:52 smithi067 bash[24488]: cluster 2024-02-10T08:38:51.122632+0000 mon.a (mon.0) 2752 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-10T08:38:52.388 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:52 smithi134 bash[18722]: cluster 2024-02-10T08:38:51.106287+0000 mon.a (mon.0) 2750 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:38:52.388 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:52 smithi134 bash[18722]: audit 2024-02-10T08:38:51.122536+0000 mon.a (mon.0) 2751 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-112"}]': finished 2024-02-10T08:38:52.388 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:52 smithi134 bash[18722]: cluster 2024-02-10T08:38:51.122632+0000 mon.a (mon.0) 2752 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-10T08:38:52.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:38:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:38:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:38:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:38:53.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:53 smithi134 bash[18722]: cluster 2024-02-10T08:38:52.128498+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-10T08:38:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:53 smithi134 bash[18722]: cluster 2024-02-10T08:38:52.238006+0000 mgr.y (mgr.14547) 636 : cluster [DBG] pgmap v1002: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:53 smithi134 bash[18722]: audit 2024-02-10T08:38:52.386954+0000 mgr.y (mgr.14547) 637 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:53 smithi067 bash[24488]: cluster 2024-02-10T08:38:52.128498+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-10T08:38:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:53 smithi067 bash[24488]: cluster 2024-02-10T08:38:52.238006+0000 mgr.y (mgr.14547) 636 : cluster [DBG] pgmap v1002: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:53.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:53 smithi067 bash[24488]: audit 2024-02-10T08:38:52.386954+0000 mgr.y (mgr.14547) 637 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:53 smithi067 bash[17385]: cluster 2024-02-10T08:38:52.128498+0000 mon.a (mon.0) 2753 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-10T08:38:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:53 smithi067 bash[17385]: cluster 2024-02-10T08:38:52.238006+0000 mgr.y (mgr.14547) 636 : cluster [DBG] pgmap v1002: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:53.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:53 smithi067 bash[17385]: audit 2024-02-10T08:38:52.386954+0000 mgr.y (mgr.14547) 637 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:38:54.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:54 smithi134 bash[18722]: cluster 2024-02-10T08:38:53.144350+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-10T08:38:54.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:54 smithi134 bash[18722]: audit 2024-02-10T08:38:53.148596+0000 mon.c (mon.2) 912 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:54.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:54 smithi134 bash[18722]: audit 2024-02-10T08:38:53.150913+0000 mon.a (mon.0) 2755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:54.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:54 smithi067 bash[17385]: cluster 2024-02-10T08:38:53.144350+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-10T08:38:54.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:54 smithi067 bash[17385]: audit 2024-02-10T08:38:53.148596+0000 mon.c (mon.2) 912 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:54.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:54 smithi067 bash[17385]: audit 2024-02-10T08:38:53.150913+0000 mon.a (mon.0) 2755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:54.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:54 smithi067 bash[24488]: cluster 2024-02-10T08:38:53.144350+0000 mon.a (mon.0) 2754 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-10T08:38:54.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:54 smithi067 bash[24488]: audit 2024-02-10T08:38:53.148596+0000 mon.c (mon.2) 912 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:54.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:54 smithi067 bash[24488]: audit 2024-02-10T08:38:53.150913+0000 mon.a (mon.0) 2755 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:38:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:55 smithi134 bash[18722]: audit 2024-02-10T08:38:54.149610+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:55 smithi134 bash[18722]: cluster 2024-02-10T08:38:54.149951+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-10T08:38:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:55 smithi134 bash[18722]: cluster 2024-02-10T08:38:54.238894+0000 mgr.y (mgr.14547) 638 : cluster [DBG] pgmap v1005: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:55 smithi134 bash[18722]: audit 2024-02-10T08:38:54.936028+0000 mon.c (mon.2) 913 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:38:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:55 smithi134 bash[18722]: cluster 2024-02-10T08:38:55.153263+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-10T08:38:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[17385]: audit 2024-02-10T08:38:54.149610+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[17385]: cluster 2024-02-10T08:38:54.149951+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-10T08:38:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[17385]: cluster 2024-02-10T08:38:54.238894+0000 mgr.y (mgr.14547) 638 : cluster [DBG] pgmap v1005: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[17385]: audit 2024-02-10T08:38:54.936028+0000 mon.c (mon.2) 913 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[17385]: cluster 2024-02-10T08:38:55.153263+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[24488]: audit 2024-02-10T08:38:54.149610+0000 mon.a (mon.0) 2756 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[24488]: cluster 2024-02-10T08:38:54.149951+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[24488]: cluster 2024-02-10T08:38:54.238894+0000 mgr.y (mgr.14547) 638 : cluster [DBG] pgmap v1005: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[24488]: audit 2024-02-10T08:38:54.936028+0000 mon.c (mon.2) 913 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:38:55.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:55 smithi067 bash[24488]: cluster 2024-02-10T08:38:55.153263+0000 mon.a (mon.0) 2758 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-10T08:38:56.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:56 smithi134 bash[18722]: audit 2024-02-10T08:38:55.186517+0000 mon.c (mon.2) 914 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:56.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:56 smithi134 bash[18722]: audit 2024-02-10T08:38:55.186964+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:56 smithi134 bash[18722]: audit 2024-02-10T08:38:55.243264+0000 mon.c (mon.2) 915 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:38:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:56 smithi134 bash[18722]: audit 2024-02-10T08:38:55.245019+0000 mon.c (mon.2) 916 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:38:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:56 smithi134 bash[18722]: audit 2024-02-10T08:38:55.256312+0000 mon.a (mon.0) 2760 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[17385]: audit 2024-02-10T08:38:55.186517+0000 mon.c (mon.2) 914 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[17385]: audit 2024-02-10T08:38:55.186964+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[17385]: audit 2024-02-10T08:38:55.243264+0000 mon.c (mon.2) 915 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:38:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[17385]: audit 2024-02-10T08:38:55.245019+0000 mon.c (mon.2) 916 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:38:56.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[17385]: audit 2024-02-10T08:38:55.256312+0000 mon.a (mon.0) 2760 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[24488]: audit 2024-02-10T08:38:55.186517+0000 mon.c (mon.2) 914 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[24488]: audit 2024-02-10T08:38:55.186964+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:38:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[24488]: audit 2024-02-10T08:38:55.243264+0000 mon.c (mon.2) 915 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:38:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[24488]: audit 2024-02-10T08:38:55.245019+0000 mon.c (mon.2) 916 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:38:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:56 smithi067 bash[24488]: audit 2024-02-10T08:38:55.256312+0000 mon.a (mon.0) 2760 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:38:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:57 smithi134 bash[18722]: audit 2024-02-10T08:38:56.187929+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:57 smithi134 bash[18722]: cluster 2024-02-10T08:38:56.188033+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-10T08:38:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:57 smithi134 bash[18722]: audit 2024-02-10T08:38:56.190500+0000 mon.c (mon.2) 917 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:38:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:57 smithi134 bash[18722]: audit 2024-02-10T08:38:56.191270+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:38:57.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:57 smithi134 bash[18722]: cluster 2024-02-10T08:38:56.239706+0000 mgr.y (mgr.14547) 639 : cluster [DBG] pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:57.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[24488]: audit 2024-02-10T08:38:56.187929+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:57.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[24488]: cluster 2024-02-10T08:38:56.188033+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-10T08:38:57.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[24488]: audit 2024-02-10T08:38:56.190500+0000 mon.c (mon.2) 917 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[24488]: audit 2024-02-10T08:38:56.191270+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[24488]: cluster 2024-02-10T08:38:56.239706+0000 mgr.y (mgr.14547) 639 : cluster [DBG] pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[17385]: audit 2024-02-10T08:38:56.187929+0000 mon.a (mon.0) 2761 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[17385]: cluster 2024-02-10T08:38:56.188033+0000 mon.a (mon.0) 2762 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[17385]: audit 2024-02-10T08:38:56.190500+0000 mon.c (mon.2) 917 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[17385]: audit 2024-02-10T08:38:56.191270+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:38:57.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[17385]: cluster 2024-02-10T08:38:56.239706+0000 mgr.y (mgr.14547) 639 : cluster [DBG] pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:38:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:38:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:38:57] "GET /metrics HTTP/1.1" 200 190662 "" "Prometheus/2.33.4" 2024-02-10T08:38:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:58 smithi134 bash[18722]: audit 2024-02-10T08:38:57.200370+0000 mon.a (mon.0) 2764 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]': finished 2024-02-10T08:38:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:58 smithi134 bash[18722]: cluster 2024-02-10T08:38:57.200484+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-10T08:38:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:58 smithi134 bash[18722]: audit 2024-02-10T08:38:57.203289+0000 mon.c (mon.2) 918 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]: dispatch 2024-02-10T08:38:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:58 smithi134 bash[18722]: audit 2024-02-10T08:38:57.205589+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]: dispatch 2024-02-10T08:38:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[17385]: audit 2024-02-10T08:38:57.200370+0000 mon.a (mon.0) 2764 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]': finished 2024-02-10T08:38:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[17385]: cluster 2024-02-10T08:38:57.200484+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-10T08:38:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[17385]: audit 2024-02-10T08:38:57.203289+0000 mon.c (mon.2) 918 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]: dispatch 2024-02-10T08:38:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[17385]: audit 2024-02-10T08:38:57.205589+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]: dispatch 2024-02-10T08:38:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[24488]: audit 2024-02-10T08:38:57.200370+0000 mon.a (mon.0) 2764 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-114"}]': finished 2024-02-10T08:38:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[24488]: cluster 2024-02-10T08:38:57.200484+0000 mon.a (mon.0) 2765 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-10T08:38:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[24488]: audit 2024-02-10T08:38:57.203289+0000 mon.c (mon.2) 918 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]: dispatch 2024-02-10T08:38:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:58 smithi067 bash[24488]: audit 2024-02-10T08:38:57.205589+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]: dispatch 2024-02-10T08:38:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:59 smithi134 bash[18722]: cluster 2024-02-10T08:38:58.195549+0000 mon.a (mon.0) 2767 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:59 smithi134 bash[18722]: audit 2024-02-10T08:38:58.204864+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]': finished 2024-02-10T08:38:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:59 smithi134 bash[18722]: cluster 2024-02-10T08:38:58.204986+0000 mon.a (mon.0) 2769 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-10T08:38:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:38:59 smithi134 bash[18722]: cluster 2024-02-10T08:38:58.240450+0000 mgr.y (mgr.14547) 640 : cluster [DBG] pgmap v1011: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:38:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[17385]: cluster 2024-02-10T08:38:58.195549+0000 mon.a (mon.0) 2767 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[17385]: audit 2024-02-10T08:38:58.204864+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]': finished 2024-02-10T08:38:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[17385]: cluster 2024-02-10T08:38:58.204986+0000 mon.a (mon.0) 2769 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-10T08:38:59.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[17385]: cluster 2024-02-10T08:38:58.240450+0000 mgr.y (mgr.14547) 640 : cluster [DBG] pgmap v1011: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:38:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[24488]: cluster 2024-02-10T08:38:58.195549+0000 mon.a (mon.0) 2767 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:38:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[24488]: audit 2024-02-10T08:38:58.204864+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-114", "mode": "writeback"}]': finished 2024-02-10T08:38:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[24488]: cluster 2024-02-10T08:38:58.204986+0000 mon.a (mon.0) 2769 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-10T08:38:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:38:59 smithi067 bash[24488]: cluster 2024-02-10T08:38:58.240450+0000 mgr.y (mgr.14547) 640 : cluster [DBG] pgmap v1011: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:39:00.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:00 smithi134 bash[18722]: cluster 2024-02-10T08:38:59.222230+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-10T08:39:00.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:00 smithi134 bash[18722]: audit 2024-02-10T08:38:59.265391+0000 mon.c (mon.2) 919 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:00.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:00 smithi134 bash[18722]: audit 2024-02-10T08:38:59.266044+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:00 smithi067 bash[17385]: cluster 2024-02-10T08:38:59.222230+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-10T08:39:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:00 smithi067 bash[17385]: audit 2024-02-10T08:38:59.265391+0000 mon.c (mon.2) 919 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:00 smithi067 bash[17385]: audit 2024-02-10T08:38:59.266044+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:00.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:00 smithi067 bash[24488]: cluster 2024-02-10T08:38:59.222230+0000 mon.a (mon.0) 2770 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-10T08:39:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:00 smithi067 bash[24488]: audit 2024-02-10T08:38:59.265391+0000 mon.c (mon.2) 919 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:00 smithi067 bash[24488]: audit 2024-02-10T08:38:59.266044+0000 mon.a (mon.0) 2771 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:01 smithi134 bash[18722]: audit 2024-02-10T08:39:00.229916+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:01 smithi134 bash[18722]: cluster 2024-02-10T08:39:00.230107+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-10T08:39:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:01 smithi134 bash[18722]: audit 2024-02-10T08:39:00.233774+0000 mon.c (mon.2) 920 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:39:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:01 smithi134 bash[18722]: audit 2024-02-10T08:39:00.234314+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:39:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:01 smithi134 bash[18722]: cluster 2024-02-10T08:39:00.240981+0000 mgr.y (mgr.14547) 641 : cluster [DBG] pgmap v1014: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:39:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:01 smithi134 bash[18722]: audit 2024-02-10T08:39:00.791255+0000 mon.a (mon.0) 2775 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[17385]: audit 2024-02-10T08:39:00.229916+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[17385]: cluster 2024-02-10T08:39:00.230107+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[17385]: audit 2024-02-10T08:39:00.233774+0000 mon.c (mon.2) 920 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[17385]: audit 2024-02-10T08:39:00.234314+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[17385]: cluster 2024-02-10T08:39:00.240981+0000 mgr.y (mgr.14547) 641 : cluster [DBG] pgmap v1014: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[17385]: audit 2024-02-10T08:39:00.791255+0000 mon.a (mon.0) 2775 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:01.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[24488]: audit 2024-02-10T08:39:00.229916+0000 mon.a (mon.0) 2772 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[24488]: cluster 2024-02-10T08:39:00.230107+0000 mon.a (mon.0) 2773 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-10T08:39:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[24488]: audit 2024-02-10T08:39:00.233774+0000 mon.c (mon.2) 920 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:39:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[24488]: audit 2024-02-10T08:39:00.234314+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]: dispatch 2024-02-10T08:39:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[24488]: cluster 2024-02-10T08:39:00.240981+0000 mgr.y (mgr.14547) 641 : cluster [DBG] pgmap v1014: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-10T08:39:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:01 smithi067 bash[24488]: audit 2024-02-10T08:39:00.791255+0000 mon.a (mon.0) 2775 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:02.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:02 smithi134 bash[18722]: cluster 2024-02-10T08:39:01.228801+0000 mon.a (mon.0) 2776 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:02.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:02 smithi134 bash[18722]: audit 2024-02-10T08:39:01.247722+0000 mon.a (mon.0) 2777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]': finished 2024-02-10T08:39:02.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:02 smithi134 bash[18722]: cluster 2024-02-10T08:39:01.247866+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-10T08:39:02.527 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:39:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:39:02.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:02 smithi067 bash[17385]: cluster 2024-02-10T08:39:01.228801+0000 mon.a (mon.0) 2776 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:02.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:02 smithi067 bash[17385]: audit 2024-02-10T08:39:01.247722+0000 mon.a (mon.0) 2777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]': finished 2024-02-10T08:39:02.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:02 smithi067 bash[17385]: cluster 2024-02-10T08:39:01.247866+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-10T08:39:02.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:02 smithi067 bash[24488]: cluster 2024-02-10T08:39:01.228801+0000 mon.a (mon.0) 2776 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:02.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:02 smithi067 bash[24488]: audit 2024-02-10T08:39:01.247722+0000 mon.a (mon.0) 2777 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-114"}]': finished 2024-02-10T08:39:02.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:02 smithi067 bash[24488]: cluster 2024-02-10T08:39:01.247866+0000 mon.a (mon.0) 2778 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-10T08:39:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:39:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:39:03.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:03 smithi134 bash[18722]: cluster 2024-02-10T08:39:02.242184+0000 mgr.y (mgr.14547) 642 : cluster [DBG] pgmap v1016: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T08:39:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:03 smithi134 bash[18722]: cluster 2024-02-10T08:39:02.250449+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-10T08:39:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:03 smithi134 bash[18722]: audit 2024-02-10T08:39:02.392183+0000 mgr.y (mgr.14547) 643 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:03 smithi067 bash[17385]: cluster 2024-02-10T08:39:02.242184+0000 mgr.y (mgr.14547) 642 : cluster [DBG] pgmap v1016: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T08:39:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:03 smithi067 bash[17385]: cluster 2024-02-10T08:39:02.250449+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-10T08:39:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:03 smithi067 bash[17385]: audit 2024-02-10T08:39:02.392183+0000 mgr.y (mgr.14547) 643 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:03 smithi067 bash[24488]: cluster 2024-02-10T08:39:02.242184+0000 mgr.y (mgr.14547) 642 : cluster [DBG] pgmap v1016: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-10T08:39:03.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:03 smithi067 bash[24488]: cluster 2024-02-10T08:39:02.250449+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-10T08:39:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:03 smithi067 bash[24488]: audit 2024-02-10T08:39:02.392183+0000 mgr.y (mgr.14547) 643 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:04.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:04 smithi134 bash[18722]: cluster 2024-02-10T08:39:03.260964+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-10T08:39:04.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:04 smithi134 bash[18722]: audit 2024-02-10T08:39:03.265303+0000 mon.c (mon.2) 921 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:04.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:04 smithi134 bash[18722]: audit 2024-02-10T08:39:03.266253+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:04 smithi067 bash[17385]: cluster 2024-02-10T08:39:03.260964+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-10T08:39:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:04 smithi067 bash[17385]: audit 2024-02-10T08:39:03.265303+0000 mon.c (mon.2) 921 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:04 smithi067 bash[17385]: audit 2024-02-10T08:39:03.266253+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:04 smithi067 bash[24488]: cluster 2024-02-10T08:39:03.260964+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-10T08:39:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:04 smithi067 bash[24488]: audit 2024-02-10T08:39:03.265303+0000 mon.c (mon.2) 921 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:04 smithi067 bash[24488]: audit 2024-02-10T08:39:03.266253+0000 mon.a (mon.0) 2781 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[17385]: cluster 2024-02-10T08:39:04.242998+0000 mgr.y (mgr.14547) 644 : cluster [DBG] pgmap v1019: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[17385]: audit 2024-02-10T08:39:04.270926+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:05.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[17385]: cluster 2024-02-10T08:39:04.271198+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[17385]: audit 2024-02-10T08:39:04.272970+0000 mon.c (mon.2) 922 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[17385]: audit 2024-02-10T08:39:04.273765+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[24488]: cluster 2024-02-10T08:39:04.242998+0000 mgr.y (mgr.14547) 644 : cluster [DBG] pgmap v1019: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[24488]: audit 2024-02-10T08:39:04.270926+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[24488]: cluster 2024-02-10T08:39:04.271198+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[24488]: audit 2024-02-10T08:39:04.272970+0000 mon.c (mon.2) 922 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:05.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:05 smithi067 bash[24488]: audit 2024-02-10T08:39:04.273765+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:05.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:05 smithi134 bash[18722]: cluster 2024-02-10T08:39:04.242998+0000 mgr.y (mgr.14547) 644 : cluster [DBG] pgmap v1019: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:05 smithi134 bash[18722]: audit 2024-02-10T08:39:04.270926+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:05 smithi134 bash[18722]: cluster 2024-02-10T08:39:04.271198+0000 mon.a (mon.0) 2783 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-10T08:39:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:05 smithi134 bash[18722]: audit 2024-02-10T08:39:04.272970+0000 mon.c (mon.2) 922 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:05.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:05 smithi134 bash[18722]: audit 2024-02-10T08:39:04.273765+0000 mon.a (mon.0) 2784 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[17385]: audit 2024-02-10T08:39:05.276523+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[17385]: cluster 2024-02-10T08:39:05.276677+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-10T08:39:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[17385]: audit 2024-02-10T08:39:05.278317+0000 mon.c (mon.2) 923 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[17385]: audit 2024-02-10T08:39:05.287437+0000 mon.a (mon.0) 2787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[24488]: audit 2024-02-10T08:39:05.276523+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:06.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[24488]: cluster 2024-02-10T08:39:05.276677+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-10T08:39:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[24488]: audit 2024-02-10T08:39:05.278317+0000 mon.c (mon.2) 923 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:06.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:06 smithi067 bash[24488]: audit 2024-02-10T08:39:05.287437+0000 mon.a (mon.0) 2787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:06 smithi134 bash[18722]: audit 2024-02-10T08:39:05.276523+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:06 smithi134 bash[18722]: cluster 2024-02-10T08:39:05.276677+0000 mon.a (mon.0) 2786 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-10T08:39:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:06 smithi134 bash[18722]: audit 2024-02-10T08:39:05.278317+0000 mon.c (mon.2) 923 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:06 smithi134 bash[18722]: audit 2024-02-10T08:39:05.287437+0000 mon.a (mon.0) 2787 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:07.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: cluster 2024-02-10T08:39:06.243819+0000 mgr.y (mgr.14547) 645 : cluster [DBG] pgmap v1022: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: audit 2024-02-10T08:39:06.286383+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]': finished 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: cluster 2024-02-10T08:39:06.286686+0000 mon.a (mon.0) 2789 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: audit 2024-02-10T08:39:06.291228+0000 mon.c (mon.2) 924 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]: dispatch 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: audit 2024-02-10T08:39:06.298567+0000 mon.a (mon.0) 2790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]: dispatch 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: cluster 2024-02-10T08:39:07.024155+0000 mon.a (mon.0) 2791 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: audit 2024-02-10T08:39:07.035707+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]': finished 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: cluster 2024-02-10T08:39:07.035833+0000 mon.a (mon.0) 2793 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: audit 2024-02-10T08:39:07.139417+0000 mon.c (mon.2) 925 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17385]: audit 2024-02-10T08:39:07.140078+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: cluster 2024-02-10T08:39:06.243819+0000 mgr.y (mgr.14547) 645 : cluster [DBG] pgmap v1022: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:07.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: audit 2024-02-10T08:39:06.286383+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]': finished 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: cluster 2024-02-10T08:39:06.286686+0000 mon.a (mon.0) 2789 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: audit 2024-02-10T08:39:06.291228+0000 mon.c (mon.2) 924 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]: dispatch 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: audit 2024-02-10T08:39:06.298567+0000 mon.a (mon.0) 2790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]: dispatch 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: cluster 2024-02-10T08:39:07.024155+0000 mon.a (mon.0) 2791 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: audit 2024-02-10T08:39:07.035707+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]': finished 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: cluster 2024-02-10T08:39:07.035833+0000 mon.a (mon.0) 2793 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: audit 2024-02-10T08:39:07.139417+0000 mon.c (mon.2) 925 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:07.622 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[24488]: audit 2024-02-10T08:39:07.140078+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:07.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: cluster 2024-02-10T08:39:06.243819+0000 mgr.y (mgr.14547) 645 : cluster [DBG] pgmap v1022: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: audit 2024-02-10T08:39:06.286383+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-116"}]': finished 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: cluster 2024-02-10T08:39:06.286686+0000 mon.a (mon.0) 2789 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: audit 2024-02-10T08:39:06.291228+0000 mon.c (mon.2) 924 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]: dispatch 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: audit 2024-02-10T08:39:06.298567+0000 mon.a (mon.0) 2790 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]: dispatch 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: cluster 2024-02-10T08:39:07.024155+0000 mon.a (mon.0) 2791 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: audit 2024-02-10T08:39:07.035707+0000 mon.a (mon.0) 2792 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-116", "mode": "writeback"}]': finished 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: cluster 2024-02-10T08:39:07.035833+0000 mon.a (mon.0) 2793 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: audit 2024-02-10T08:39:07.139417+0000 mon.c (mon.2) 925 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:07 smithi134 bash[18722]: audit 2024-02-10T08:39:07.140078+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:39:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:07] "GET /metrics HTTP/1.1" 200 190786 "" "Prometheus/2.33.4" 2024-02-10T08:39:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[24488]: audit 2024-02-10T08:39:08.048372+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[24488]: cluster 2024-02-10T08:39:08.048496+0000 mon.a (mon.0) 2796 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-10T08:39:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[24488]: audit 2024-02-10T08:39:08.052700+0000 mon.c (mon.2) 926 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[24488]: audit 2024-02-10T08:39:08.053190+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[24488]: cluster 2024-02-10T08:39:08.244690+0000 mgr.y (mgr.14547) 646 : cluster [DBG] pgmap v1026: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:39:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[17385]: audit 2024-02-10T08:39:08.048372+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[17385]: cluster 2024-02-10T08:39:08.048496+0000 mon.a (mon.0) 2796 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-10T08:39:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[17385]: audit 2024-02-10T08:39:08.052700+0000 mon.c (mon.2) 926 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[17385]: audit 2024-02-10T08:39:08.053190+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:09.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:09 smithi067 bash[17385]: cluster 2024-02-10T08:39:08.244690+0000 mgr.y (mgr.14547) 646 : cluster [DBG] pgmap v1026: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:39:09.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:09 smithi134 bash[18722]: audit 2024-02-10T08:39:08.048372+0000 mon.a (mon.0) 2795 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:09.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:09 smithi134 bash[18722]: cluster 2024-02-10T08:39:08.048496+0000 mon.a (mon.0) 2796 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-10T08:39:09.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:09 smithi134 bash[18722]: audit 2024-02-10T08:39:08.052700+0000 mon.c (mon.2) 926 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:09.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:09 smithi134 bash[18722]: audit 2024-02-10T08:39:08.053190+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]: dispatch 2024-02-10T08:39:09.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:09 smithi134 bash[18722]: cluster 2024-02-10T08:39:08.244690+0000 mgr.y (mgr.14547) 646 : cluster [DBG] pgmap v1026: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:39:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:10 smithi067 bash[17385]: cluster 2024-02-10T08:39:09.043118+0000 mon.a (mon.0) 2798 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:10 smithi067 bash[17385]: audit 2024-02-10T08:39:09.051103+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]': finished 2024-02-10T08:39:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:10 smithi067 bash[17385]: cluster 2024-02-10T08:39:09.051191+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-10T08:39:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:10 smithi067 bash[24488]: cluster 2024-02-10T08:39:09.043118+0000 mon.a (mon.0) 2798 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:10 smithi067 bash[24488]: audit 2024-02-10T08:39:09.051103+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]': finished 2024-02-10T08:39:10.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:10 smithi067 bash[24488]: cluster 2024-02-10T08:39:09.051191+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-10T08:39:10.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:10 smithi134 bash[18722]: cluster 2024-02-10T08:39:09.043118+0000 mon.a (mon.0) 2798 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:10.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:10 smithi134 bash[18722]: audit 2024-02-10T08:39:09.051103+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-116"}]': finished 2024-02-10T08:39:10.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:10 smithi134 bash[18722]: cluster 2024-02-10T08:39:09.051191+0000 mon.a (mon.0) 2800 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-10T08:39:11.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:11 smithi067 bash[17385]: cluster 2024-02-10T08:39:10.067957+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-10T08:39:11.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:11 smithi067 bash[17385]: cluster 2024-02-10T08:39:10.245486+0000 mgr.y (mgr.14547) 647 : cluster [DBG] pgmap v1029: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:39:11.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:11 smithi067 bash[24488]: cluster 2024-02-10T08:39:10.067957+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-10T08:39:11.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:11 smithi067 bash[24488]: cluster 2024-02-10T08:39:10.245486+0000 mgr.y (mgr.14547) 647 : cluster [DBG] pgmap v1029: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:39:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:11 smithi134 bash[18722]: cluster 2024-02-10T08:39:10.067957+0000 mon.a (mon.0) 2801 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-10T08:39:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:11 smithi134 bash[18722]: cluster 2024-02-10T08:39:10.245486+0000 mgr.y (mgr.14547) 647 : cluster [DBG] pgmap v1029: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-10T08:39:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:12 smithi067 bash[24488]: cluster 2024-02-10T08:39:11.079964+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-10T08:39:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:12 smithi067 bash[24488]: audit 2024-02-10T08:39:11.083264+0000 mon.c (mon.2) 927 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:12 smithi067 bash[24488]: audit 2024-02-10T08:39:11.085389+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:12.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:12 smithi067 bash[17385]: cluster 2024-02-10T08:39:11.079964+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-10T08:39:12.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:12 smithi067 bash[17385]: audit 2024-02-10T08:39:11.083264+0000 mon.c (mon.2) 927 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:12.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:12 smithi067 bash[17385]: audit 2024-02-10T08:39:11.085389+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:12.404 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:12 smithi134 bash[18722]: cluster 2024-02-10T08:39:11.079964+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-10T08:39:12.405 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:12 smithi134 bash[18722]: audit 2024-02-10T08:39:11.083264+0000 mon.c (mon.2) 927 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:12.405 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:12 smithi134 bash[18722]: audit 2024-02-10T08:39:11.085389+0000 mon.a (mon.0) 2803 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:12.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:39:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:39:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[24488]: audit 2024-02-10T08:39:12.084759+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[24488]: cluster 2024-02-10T08:39:12.085302+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-10T08:39:13.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[24488]: audit 2024-02-10T08:39:12.100801+0000 mon.c (mon.2) 928 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[24488]: audit 2024-02-10T08:39:12.103178+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[24488]: cluster 2024-02-10T08:39:12.246361+0000 mgr.y (mgr.14547) 648 : cluster [DBG] pgmap v1032: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[24488]: audit 2024-02-10T08:39:12.403732+0000 mgr.y (mgr.14547) 649 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[17385]: audit 2024-02-10T08:39:12.084759+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[17385]: cluster 2024-02-10T08:39:12.085302+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[17385]: audit 2024-02-10T08:39:12.100801+0000 mon.c (mon.2) 928 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[17385]: audit 2024-02-10T08:39:12.103178+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[17385]: cluster 2024-02-10T08:39:12.246361+0000 mgr.y (mgr.14547) 648 : cluster [DBG] pgmap v1032: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:13.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:13 smithi067 bash[17385]: audit 2024-02-10T08:39:12.403732+0000 mgr.y (mgr.14547) 649 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:39:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[18722]: audit 2024-02-10T08:39:12.084759+0000 mon.a (mon.0) 2804 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[18722]: cluster 2024-02-10T08:39:12.085302+0000 mon.a (mon.0) 2805 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-10T08:39:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[18722]: audit 2024-02-10T08:39:12.100801+0000 mon.c (mon.2) 928 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[18722]: audit 2024-02-10T08:39:12.103178+0000 mon.a (mon.0) 2806 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[18722]: cluster 2024-02-10T08:39:12.246361+0000 mgr.y (mgr.14547) 648 : cluster [DBG] pgmap v1032: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:13 smithi134 bash[18722]: audit 2024-02-10T08:39:12.403732+0000 mgr.y (mgr.14547) 649 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[17385]: audit 2024-02-10T08:39:13.101517+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[17385]: cluster 2024-02-10T08:39:13.101692+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-10T08:39:14.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[17385]: audit 2024-02-10T08:39:13.103030+0000 mon.c (mon.2) 929 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[17385]: audit 2024-02-10T08:39:13.104115+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[17385]: audit 2024-02-10T08:39:14.102571+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]': finished 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[17385]: cluster 2024-02-10T08:39:14.102785+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[24488]: audit 2024-02-10T08:39:13.101517+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[24488]: cluster 2024-02-10T08:39:13.101692+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[24488]: audit 2024-02-10T08:39:13.103030+0000 mon.c (mon.2) 929 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[24488]: audit 2024-02-10T08:39:13.104115+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:14.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[24488]: audit 2024-02-10T08:39:14.102571+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]': finished 2024-02-10T08:39:14.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:14 smithi067 bash[24488]: cluster 2024-02-10T08:39:14.102785+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-10T08:39:14.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:14 smithi134 bash[18722]: audit 2024-02-10T08:39:13.101517+0000 mon.a (mon.0) 2807 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:14.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:14 smithi134 bash[18722]: cluster 2024-02-10T08:39:13.101692+0000 mon.a (mon.0) 2808 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-10T08:39:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:14 smithi134 bash[18722]: audit 2024-02-10T08:39:13.103030+0000 mon.c (mon.2) 929 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:14 smithi134 bash[18722]: audit 2024-02-10T08:39:13.104115+0000 mon.a (mon.0) 2809 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:14 smithi134 bash[18722]: audit 2024-02-10T08:39:14.102571+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-118"}]': finished 2024-02-10T08:39:14.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:14 smithi134 bash[18722]: cluster 2024-02-10T08:39:14.102785+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-10T08:39:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[24488]: audit 2024-02-10T08:39:14.106214+0000 mon.c (mon.2) 930 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]: dispatch 2024-02-10T08:39:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[24488]: audit 2024-02-10T08:39:14.106933+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]: dispatch 2024-02-10T08:39:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[24488]: cluster 2024-02-10T08:39:14.247235+0000 mgr.y (mgr.14547) 650 : cluster [DBG] pgmap v1035: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[24488]: cluster 2024-02-10T08:39:15.096840+0000 mon.a (mon.0) 2813 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[24488]: audit 2024-02-10T08:39:15.105680+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]': finished 2024-02-10T08:39:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[24488]: cluster 2024-02-10T08:39:15.105937+0000 mon.a (mon.0) 2815 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-10T08:39:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[17385]: audit 2024-02-10T08:39:14.106214+0000 mon.c (mon.2) 930 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]: dispatch 2024-02-10T08:39:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[17385]: audit 2024-02-10T08:39:14.106933+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]: dispatch 2024-02-10T08:39:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[17385]: cluster 2024-02-10T08:39:14.247235+0000 mgr.y (mgr.14547) 650 : cluster [DBG] pgmap v1035: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[17385]: cluster 2024-02-10T08:39:15.096840+0000 mon.a (mon.0) 2813 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[17385]: audit 2024-02-10T08:39:15.105680+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]': finished 2024-02-10T08:39:15.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:15 smithi067 bash[17385]: cluster 2024-02-10T08:39:15.105937+0000 mon.a (mon.0) 2815 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-10T08:39:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:15 smithi134 bash[18722]: audit 2024-02-10T08:39:14.106214+0000 mon.c (mon.2) 930 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]: dispatch 2024-02-10T08:39:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:15 smithi134 bash[18722]: audit 2024-02-10T08:39:14.106933+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]: dispatch 2024-02-10T08:39:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:15 smithi134 bash[18722]: cluster 2024-02-10T08:39:14.247235+0000 mgr.y (mgr.14547) 650 : cluster [DBG] pgmap v1035: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:15 smithi134 bash[18722]: cluster 2024-02-10T08:39:15.096840+0000 mon.a (mon.0) 2813 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:15 smithi134 bash[18722]: audit 2024-02-10T08:39:15.105680+0000 mon.a (mon.0) 2814 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-118", "mode": "writeback"}]': finished 2024-02-10T08:39:15.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:15 smithi134 bash[18722]: cluster 2024-02-10T08:39:15.105937+0000 mon.a (mon.0) 2815 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-10T08:39:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:17 smithi067 bash[17385]: cluster 2024-02-10T08:39:16.248006+0000 mgr.y (mgr.14547) 651 : cluster [DBG] pgmap v1037: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:17.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:17 smithi067 bash[24488]: cluster 2024-02-10T08:39:16.248006+0000 mgr.y (mgr.14547) 651 : cluster [DBG] pgmap v1037: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:17.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:17 smithi134 bash[18722]: cluster 2024-02-10T08:39:16.248006+0000 mgr.y (mgr.14547) 651 : cluster [DBG] pgmap v1037: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:17.776 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:39:17 smithi134 bash[41763]: t=2024-02-10T08:39:17+0000 lvl=info msg="Database locked, sleeping then retrying" logger=sqlstore error="database is locked" retry=0 2024-02-10T08:39:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:39:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:17] "GET /metrics HTTP/1.1" 200 190916 "" "Prometheus/2.33.4" 2024-02-10T08:39:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:19 smithi067 bash[17385]: cluster 2024-02-10T08:39:18.250186+0000 mgr.y (mgr.14547) 652 : cluster [DBG] pgmap v1038: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-10T08:39:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:19 smithi067 bash[24488]: cluster 2024-02-10T08:39:18.250186+0000 mgr.y (mgr.14547) 652 : cluster [DBG] pgmap v1038: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-10T08:39:19.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:19 smithi134 bash[18722]: cluster 2024-02-10T08:39:18.250186+0000 mgr.y (mgr.14547) 652 : cluster [DBG] pgmap v1038: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-10T08:39:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:20 smithi067 bash[17385]: audit 2024-02-10T08:39:20.223158+0000 mon.c (mon.2) 931 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:20.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:20 smithi067 bash[17385]: audit 2024-02-10T08:39:20.224084+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:20 smithi067 bash[24488]: audit 2024-02-10T08:39:20.223158+0000 mon.c (mon.2) 931 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:20.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:20 smithi067 bash[24488]: audit 2024-02-10T08:39:20.224084+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:20.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:20 smithi134 bash[18722]: audit 2024-02-10T08:39:20.223158+0000 mon.c (mon.2) 931 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:20.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:20 smithi134 bash[18722]: audit 2024-02-10T08:39:20.224084+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[17385]: cluster 2024-02-10T08:39:20.251063+0000 mgr.y (mgr.14547) 653 : cluster [DBG] pgmap v1039: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 858 B/s rd, 286 B/s wr, 1 op/s 2024-02-10T08:39:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[17385]: audit 2024-02-10T08:39:20.323169+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[17385]: cluster 2024-02-10T08:39:20.323232+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-10T08:39:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[17385]: audit 2024-02-10T08:39:20.328210+0000 mon.c (mon.2) 932 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[17385]: audit 2024-02-10T08:39:20.328900+0000 mon.a (mon.0) 2819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[24488]: cluster 2024-02-10T08:39:20.251063+0000 mgr.y (mgr.14547) 653 : cluster [DBG] pgmap v1039: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 858 B/s rd, 286 B/s wr, 1 op/s 2024-02-10T08:39:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[24488]: audit 2024-02-10T08:39:20.323169+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[24488]: cluster 2024-02-10T08:39:20.323232+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-10T08:39:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[24488]: audit 2024-02-10T08:39:20.328210+0000 mon.c (mon.2) 932 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:21.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:21 smithi067 bash[24488]: audit 2024-02-10T08:39:20.328900+0000 mon.a (mon.0) 2819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:21.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:21 smithi134 bash[18722]: cluster 2024-02-10T08:39:20.251063+0000 mgr.y (mgr.14547) 653 : cluster [DBG] pgmap v1039: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 858 B/s rd, 286 B/s wr, 1 op/s 2024-02-10T08:39:21.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:21 smithi134 bash[18722]: audit 2024-02-10T08:39:20.323169+0000 mon.a (mon.0) 2817 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:21.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:21 smithi134 bash[18722]: cluster 2024-02-10T08:39:20.323232+0000 mon.a (mon.0) 2818 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-10T08:39:21.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:21 smithi134 bash[18722]: audit 2024-02-10T08:39:20.328210+0000 mon.c (mon.2) 932 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:21.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:21 smithi134 bash[18722]: audit 2024-02-10T08:39:20.328900+0000 mon.a (mon.0) 2819 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]: dispatch 2024-02-10T08:39:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:22 smithi067 bash[17385]: cluster 2024-02-10T08:39:21.319688+0000 mon.a (mon.0) 2820 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:22 smithi067 bash[17385]: audit 2024-02-10T08:39:21.328035+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]': finished 2024-02-10T08:39:22.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:22 smithi067 bash[17385]: cluster 2024-02-10T08:39:21.328242+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-10T08:39:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:22 smithi067 bash[24488]: cluster 2024-02-10T08:39:21.319688+0000 mon.a (mon.0) 2820 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:22 smithi067 bash[24488]: audit 2024-02-10T08:39:21.328035+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]': finished 2024-02-10T08:39:22.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:22 smithi067 bash[24488]: cluster 2024-02-10T08:39:21.328242+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-10T08:39:22.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:39:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:39:22.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:22 smithi134 bash[18722]: cluster 2024-02-10T08:39:21.319688+0000 mon.a (mon.0) 2820 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:22.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:22 smithi134 bash[18722]: audit 2024-02-10T08:39:21.328035+0000 mon.a (mon.0) 2821 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-118"}]': finished 2024-02-10T08:39:22.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:22 smithi134 bash[18722]: cluster 2024-02-10T08:39:21.328242+0000 mon.a (mon.0) 2822 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-10T08:39:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:39:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:39:23.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:23 smithi134 bash[18722]: cluster 2024-02-10T08:39:22.252993+0000 mgr.y (mgr.14547) 654 : cluster [DBG] pgmap v1042: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 286 B/s wr, 2 op/s 2024-02-10T08:39:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:23 smithi134 bash[18722]: cluster 2024-02-10T08:39:22.342566+0000 mon.a (mon.0) 2823 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-10T08:39:23.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:23 smithi134 bash[18722]: audit 2024-02-10T08:39:22.415115+0000 mgr.y (mgr.14547) 655 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:23 smithi067 bash[24488]: cluster 2024-02-10T08:39:22.252993+0000 mgr.y (mgr.14547) 654 : cluster [DBG] pgmap v1042: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 286 B/s wr, 2 op/s 2024-02-10T08:39:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:23 smithi067 bash[24488]: cluster 2024-02-10T08:39:22.342566+0000 mon.a (mon.0) 2823 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-10T08:39:23.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:23 smithi067 bash[24488]: audit 2024-02-10T08:39:22.415115+0000 mgr.y (mgr.14547) 655 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:23 smithi067 bash[17385]: cluster 2024-02-10T08:39:22.252993+0000 mgr.y (mgr.14547) 654 : cluster [DBG] pgmap v1042: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 286 B/s wr, 2 op/s 2024-02-10T08:39:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:23 smithi067 bash[17385]: cluster 2024-02-10T08:39:22.342566+0000 mon.a (mon.0) 2823 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-10T08:39:23.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:23 smithi067 bash[17385]: audit 2024-02-10T08:39:22.415115+0000 mgr.y (mgr.14547) 655 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:24.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:24 smithi067 bash[17385]: cluster 2024-02-10T08:39:23.361978+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-10T08:39:24.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:24 smithi067 bash[17385]: audit 2024-02-10T08:39:23.368472+0000 mon.c (mon.2) 933 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:24.624 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:24 smithi067 bash[17385]: audit 2024-02-10T08:39:23.372907+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:24.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:24 smithi067 bash[24488]: cluster 2024-02-10T08:39:23.361978+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-10T08:39:24.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:24 smithi067 bash[24488]: audit 2024-02-10T08:39:23.368472+0000 mon.c (mon.2) 933 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:24.624 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:24 smithi067 bash[24488]: audit 2024-02-10T08:39:23.372907+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:24.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:24 smithi134 bash[18722]: cluster 2024-02-10T08:39:23.361978+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-10T08:39:24.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:24 smithi134 bash[18722]: audit 2024-02-10T08:39:23.368472+0000 mon.c (mon.2) 933 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:24.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:24 smithi134 bash[18722]: audit 2024-02-10T08:39:23.372907+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:25 smithi134 bash[18722]: cluster 2024-02-10T08:39:24.253828+0000 mgr.y (mgr.14547) 656 : cluster [DBG] pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:39:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:25 smithi134 bash[18722]: audit 2024-02-10T08:39:24.371830+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:25 smithi134 bash[18722]: cluster 2024-02-10T08:39:24.372053+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-10T08:39:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:25 smithi134 bash[18722]: audit 2024-02-10T08:39:24.376015+0000 mon.c (mon.2) 934 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:25.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:25 smithi134 bash[18722]: audit 2024-02-10T08:39:24.378058+0000 mon.a (mon.0) 2828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[17385]: cluster 2024-02-10T08:39:24.253828+0000 mgr.y (mgr.14547) 656 : cluster [DBG] pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:39:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[17385]: audit 2024-02-10T08:39:24.371830+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[17385]: cluster 2024-02-10T08:39:24.372053+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[17385]: audit 2024-02-10T08:39:24.376015+0000 mon.c (mon.2) 934 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[17385]: audit 2024-02-10T08:39:24.378058+0000 mon.a (mon.0) 2828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[24488]: cluster 2024-02-10T08:39:24.253828+0000 mgr.y (mgr.14547) 656 : cluster [DBG] pgmap v1045: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[24488]: audit 2024-02-10T08:39:24.371830+0000 mon.a (mon.0) 2826 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[24488]: cluster 2024-02-10T08:39:24.372053+0000 mon.a (mon.0) 2827 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[24488]: audit 2024-02-10T08:39:24.376015+0000 mon.c (mon.2) 934 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:25.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:25 smithi067 bash[24488]: audit 2024-02-10T08:39:24.378058+0000 mon.a (mon.0) 2828 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:26.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:26 smithi134 bash[18722]: audit 2024-02-10T08:39:25.377174+0000 mon.a (mon.0) 2829 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:26.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:26 smithi134 bash[18722]: cluster 2024-02-10T08:39:25.377286+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-10T08:39:26.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:26 smithi134 bash[18722]: audit 2024-02-10T08:39:25.378975+0000 mon.c (mon.2) 935 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:26.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:26 smithi134 bash[18722]: audit 2024-02-10T08:39:25.384404+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:26.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[17385]: audit 2024-02-10T08:39:25.377174+0000 mon.a (mon.0) 2829 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:26.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[17385]: cluster 2024-02-10T08:39:25.377286+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-10T08:39:26.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[17385]: audit 2024-02-10T08:39:25.378975+0000 mon.c (mon.2) 935 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:26.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[17385]: audit 2024-02-10T08:39:25.384404+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:26.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[24488]: audit 2024-02-10T08:39:25.377174+0000 mon.a (mon.0) 2829 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:26.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[24488]: cluster 2024-02-10T08:39:25.377286+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-10T08:39:26.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[24488]: audit 2024-02-10T08:39:25.378975+0000 mon.c (mon.2) 935 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:26.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:26 smithi067 bash[24488]: audit 2024-02-10T08:39:25.384404+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:27.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:27 smithi134 bash[18722]: cluster 2024-02-10T08:39:26.254634+0000 mgr.y (mgr.14547) 657 : cluster [DBG] pgmap v1048: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:27.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:27 smithi134 bash[18722]: audit 2024-02-10T08:39:26.393810+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]': finished 2024-02-10T08:39:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:27 smithi134 bash[18722]: cluster 2024-02-10T08:39:26.393920+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-10T08:39:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:27 smithi134 bash[18722]: audit 2024-02-10T08:39:26.396290+0000 mon.c (mon.2) 936 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]: dispatch 2024-02-10T08:39:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:27 smithi134 bash[18722]: audit 2024-02-10T08:39:26.397816+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]: dispatch 2024-02-10T08:39:27.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[24488]: cluster 2024-02-10T08:39:26.254634+0000 mgr.y (mgr.14547) 657 : cluster [DBG] pgmap v1048: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[24488]: audit 2024-02-10T08:39:26.393810+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]': finished 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[24488]: cluster 2024-02-10T08:39:26.393920+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[24488]: audit 2024-02-10T08:39:26.396290+0000 mon.c (mon.2) 936 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]: dispatch 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[24488]: audit 2024-02-10T08:39:26.397816+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]: dispatch 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[17385]: cluster 2024-02-10T08:39:26.254634+0000 mgr.y (mgr.14547) 657 : cluster [DBG] pgmap v1048: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[17385]: audit 2024-02-10T08:39:26.393810+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-120"}]': finished 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[17385]: cluster 2024-02-10T08:39:26.393920+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[17385]: audit 2024-02-10T08:39:26.396290+0000 mon.c (mon.2) 936 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]: dispatch 2024-02-10T08:39:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[17385]: audit 2024-02-10T08:39:26.397816+0000 mon.a (mon.0) 2834 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]: dispatch 2024-02-10T08:39:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:39:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:27] "GET /metrics HTTP/1.1" 200 190916 "" "Prometheus/2.33.4" 2024-02-10T08:39:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:28 smithi134 bash[18722]: cluster 2024-02-10T08:39:27.390898+0000 mon.a (mon.0) 2835 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:28 smithi134 bash[18722]: audit 2024-02-10T08:39:27.400742+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]': finished 2024-02-10T08:39:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:28 smithi134 bash[18722]: cluster 2024-02-10T08:39:27.400989+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-10T08:39:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:28 smithi134 bash[18722]: audit 2024-02-10T08:39:27.558308+0000 mon.c (mon.2) 937 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:28.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:28 smithi134 bash[18722]: audit 2024-02-10T08:39:27.559134+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[17385]: cluster 2024-02-10T08:39:27.390898+0000 mon.a (mon.0) 2835 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[17385]: audit 2024-02-10T08:39:27.400742+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]': finished 2024-02-10T08:39:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[17385]: cluster 2024-02-10T08:39:27.400989+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-10T08:39:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[17385]: audit 2024-02-10T08:39:27.558308+0000 mon.c (mon.2) 937 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:28.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[17385]: audit 2024-02-10T08:39:27.559134+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:28.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[24488]: cluster 2024-02-10T08:39:27.390898+0000 mon.a (mon.0) 2835 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:28.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[24488]: audit 2024-02-10T08:39:27.400742+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-120", "mode": "writeback"}]': finished 2024-02-10T08:39:28.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[24488]: cluster 2024-02-10T08:39:27.400989+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-10T08:39:28.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[24488]: audit 2024-02-10T08:39:27.558308+0000 mon.c (mon.2) 937 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:28.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:28 smithi067 bash[24488]: audit 2024-02-10T08:39:27.559134+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:29 smithi134 bash[18722]: cluster 2024-02-10T08:39:28.255493+0000 mgr.y (mgr.14547) 658 : cluster [DBG] pgmap v1051: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:29 smithi134 bash[18722]: audit 2024-02-10T08:39:28.418571+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:29 smithi134 bash[18722]: cluster 2024-02-10T08:39:28.418701+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-10T08:39:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:29 smithi134 bash[18722]: audit 2024-02-10T08:39:28.424571+0000 mon.c (mon.2) 938 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:29.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:29 smithi134 bash[18722]: audit 2024-02-10T08:39:28.425096+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[17385]: cluster 2024-02-10T08:39:28.255493+0000 mgr.y (mgr.14547) 658 : cluster [DBG] pgmap v1051: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[17385]: audit 2024-02-10T08:39:28.418571+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[17385]: cluster 2024-02-10T08:39:28.418701+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[17385]: audit 2024-02-10T08:39:28.424571+0000 mon.c (mon.2) 938 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[17385]: audit 2024-02-10T08:39:28.425096+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[24488]: cluster 2024-02-10T08:39:28.255493+0000 mgr.y (mgr.14547) 658 : cluster [DBG] pgmap v1051: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[24488]: audit 2024-02-10T08:39:28.418571+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[24488]: cluster 2024-02-10T08:39:28.418701+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[24488]: audit 2024-02-10T08:39:28.424571+0000 mon.c (mon.2) 938 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:29.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:29 smithi067 bash[24488]: audit 2024-02-10T08:39:28.425096+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]: dispatch 2024-02-10T08:39:30.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:30 smithi134 bash[18722]: cluster 2024-02-10T08:39:29.416474+0000 mon.a (mon.0) 2842 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:30 smithi134 bash[18722]: audit 2024-02-10T08:39:29.425516+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]': finished 2024-02-10T08:39:30.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:30 smithi134 bash[18722]: cluster 2024-02-10T08:39:29.425727+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-10T08:39:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:30 smithi067 bash[17385]: cluster 2024-02-10T08:39:29.416474+0000 mon.a (mon.0) 2842 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:30 smithi067 bash[17385]: audit 2024-02-10T08:39:29.425516+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]': finished 2024-02-10T08:39:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:30 smithi067 bash[17385]: cluster 2024-02-10T08:39:29.425727+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-10T08:39:30.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:30 smithi067 bash[24488]: cluster 2024-02-10T08:39:29.416474+0000 mon.a (mon.0) 2842 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:30.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:30 smithi067 bash[24488]: audit 2024-02-10T08:39:29.425516+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-120"}]': finished 2024-02-10T08:39:30.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:30 smithi067 bash[24488]: cluster 2024-02-10T08:39:29.425727+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-10T08:39:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:31 smithi134 bash[18722]: cluster 2024-02-10T08:39:30.256384+0000 mgr.y (mgr.14547) 659 : cluster [DBG] pgmap v1054: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:31 smithi134 bash[18722]: cluster 2024-02-10T08:39:30.448028+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-10T08:39:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:31 smithi134 bash[18722]: audit 2024-02-10T08:39:30.920268+0000 mon.a (mon.0) 2846 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:31 smithi067 bash[17385]: cluster 2024-02-10T08:39:30.256384+0000 mgr.y (mgr.14547) 659 : cluster [DBG] pgmap v1054: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:31 smithi067 bash[17385]: cluster 2024-02-10T08:39:30.448028+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-10T08:39:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:31 smithi067 bash[17385]: audit 2024-02-10T08:39:30.920268+0000 mon.a (mon.0) 2846 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:31 smithi067 bash[24488]: cluster 2024-02-10T08:39:30.256384+0000 mgr.y (mgr.14547) 659 : cluster [DBG] pgmap v1054: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:39:31.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:31 smithi067 bash[24488]: cluster 2024-02-10T08:39:30.448028+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-10T08:39:31.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:31 smithi067 bash[24488]: audit 2024-02-10T08:39:30.920268+0000 mon.a (mon.0) 2846 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:32.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:39:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:39:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:32 smithi134 bash[18722]: cluster 2024-02-10T08:39:31.458461+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-10T08:39:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:32 smithi134 bash[18722]: audit 2024-02-10T08:39:31.463151+0000 mon.c (mon.2) 939 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:32 smithi134 bash[18722]: audit 2024-02-10T08:39:31.463910+0000 mon.a (mon.0) 2848 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:32.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:32 smithi067 bash[17385]: cluster 2024-02-10T08:39:31.458461+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-10T08:39:32.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:32 smithi067 bash[17385]: audit 2024-02-10T08:39:31.463151+0000 mon.c (mon.2) 939 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:32.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:32 smithi067 bash[17385]: audit 2024-02-10T08:39:31.463910+0000 mon.a (mon.0) 2848 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:32.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:32 smithi067 bash[24488]: cluster 2024-02-10T08:39:31.458461+0000 mon.a (mon.0) 2847 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-10T08:39:32.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:32 smithi067 bash[24488]: audit 2024-02-10T08:39:31.463151+0000 mon.c (mon.2) 939 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:32.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:32 smithi067 bash[24488]: audit 2024-02-10T08:39:31.463910+0000 mon.a (mon.0) 2848 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:33.483 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:39:33.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[18722]: cluster 2024-02-10T08:39:32.258445+0000 mgr.y (mgr.14547) 660 : cluster [DBG] pgmap v1057: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:33.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[18722]: audit 2024-02-10T08:39:32.426532+0000 mgr.y (mgr.14547) 661 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:33.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[18722]: audit 2024-02-10T08:39:32.473269+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[18722]: cluster 2024-02-10T08:39:32.473522+0000 mon.a (mon.0) 2850 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-10T08:39:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[18722]: audit 2024-02-10T08:39:32.478504+0000 mon.c (mon.2) 940 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:33 smithi134 bash[18722]: audit 2024-02-10T08:39:32.479775+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[17385]: cluster 2024-02-10T08:39:32.258445+0000 mgr.y (mgr.14547) 660 : cluster [DBG] pgmap v1057: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[17385]: audit 2024-02-10T08:39:32.426532+0000 mgr.y (mgr.14547) 661 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[17385]: audit 2024-02-10T08:39:32.473269+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[17385]: cluster 2024-02-10T08:39:32.473522+0000 mon.a (mon.0) 2850 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[17385]: audit 2024-02-10T08:39:32.478504+0000 mon.c (mon.2) 940 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[17385]: audit 2024-02-10T08:39:32.479775+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[24488]: cluster 2024-02-10T08:39:32.258445+0000 mgr.y (mgr.14547) 660 : cluster [DBG] pgmap v1057: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[24488]: audit 2024-02-10T08:39:32.426532+0000 mgr.y (mgr.14547) 661 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[24488]: audit 2024-02-10T08:39:32.473269+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[24488]: cluster 2024-02-10T08:39:32.473522+0000 mon.a (mon.0) 2850 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[24488]: audit 2024-02-10T08:39:32.478504+0000 mon.c (mon.2) 940 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:33 smithi067 bash[24488]: audit 2024-02-10T08:39:32.479775+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:34 smithi134 bash[18722]: audit 2024-02-10T08:39:33.473883+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:34 smithi134 bash[18722]: cluster 2024-02-10T08:39:33.474063+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-10T08:39:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:34 smithi134 bash[18722]: audit 2024-02-10T08:39:33.477760+0000 mon.c (mon.2) 941 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:34.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:34 smithi134 bash[18722]: audit 2024-02-10T08:39:33.484592+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[17385]: audit 2024-02-10T08:39:33.473883+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[17385]: cluster 2024-02-10T08:39:33.474063+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-10T08:39:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[17385]: audit 2024-02-10T08:39:33.477760+0000 mon.c (mon.2) 941 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[17385]: audit 2024-02-10T08:39:33.484592+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:34.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[24488]: audit 2024-02-10T08:39:33.473883+0000 mon.a (mon.0) 2852 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:34.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[24488]: cluster 2024-02-10T08:39:33.474063+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-10T08:39:34.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[24488]: audit 2024-02-10T08:39:33.477760+0000 mon.c (mon.2) 941 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:34.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:34 smithi067 bash[24488]: audit 2024-02-10T08:39:33.484592+0000 mon.a (mon.0) 2854 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:35 smithi134 bash[18722]: cluster 2024-02-10T08:39:34.259325+0000 mgr.y (mgr.14547) 662 : cluster [DBG] pgmap v1060: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:35 smithi134 bash[18722]: audit 2024-02-10T08:39:34.490248+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:35 smithi134 bash[18722]: cluster 2024-02-10T08:39:34.490429+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-10T08:39:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:35 smithi134 bash[18722]: audit 2024-02-10T08:39:34.493032+0000 mon.c (mon.2) 942 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]: dispatch 2024-02-10T08:39:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:35 smithi134 bash[18722]: audit 2024-02-10T08:39:34.499737+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]: dispatch 2024-02-10T08:39:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[17385]: cluster 2024-02-10T08:39:34.259325+0000 mgr.y (mgr.14547) 662 : cluster [DBG] pgmap v1060: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[17385]: audit 2024-02-10T08:39:34.490248+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[17385]: cluster 2024-02-10T08:39:34.490429+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[17385]: audit 2024-02-10T08:39:34.493032+0000 mon.c (mon.2) 942 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]: dispatch 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[17385]: audit 2024-02-10T08:39:34.499737+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]: dispatch 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[24488]: cluster 2024-02-10T08:39:34.259325+0000 mgr.y (mgr.14547) 662 : cluster [DBG] pgmap v1060: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[24488]: audit 2024-02-10T08:39:34.490248+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[24488]: cluster 2024-02-10T08:39:34.490429+0000 mon.a (mon.0) 2856 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[24488]: audit 2024-02-10T08:39:34.493032+0000 mon.c (mon.2) 942 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]: dispatch 2024-02-10T08:39:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:35 smithi067 bash[24488]: audit 2024-02-10T08:39:34.499737+0000 mon.a (mon.0) 2857 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]: dispatch 2024-02-10T08:39:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:36 smithi134 bash[18722]: cluster 2024-02-10T08:39:35.494318+0000 mon.a (mon.0) 2858 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:36 smithi134 bash[18722]: audit 2024-02-10T08:39:35.502402+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]': finished 2024-02-10T08:39:36.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:36 smithi134 bash[18722]: cluster 2024-02-10T08:39:35.502541+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-10T08:39:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:36 smithi067 bash[17385]: cluster 2024-02-10T08:39:35.494318+0000 mon.a (mon.0) 2858 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:36 smithi067 bash[17385]: audit 2024-02-10T08:39:35.502402+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]': finished 2024-02-10T08:39:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:36 smithi067 bash[17385]: cluster 2024-02-10T08:39:35.502541+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-10T08:39:36.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:36 smithi067 bash[24488]: cluster 2024-02-10T08:39:35.494318+0000 mon.a (mon.0) 2858 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:36.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:36 smithi067 bash[24488]: audit 2024-02-10T08:39:35.502402+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-122", "mode": "writeback"}]': finished 2024-02-10T08:39:36.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:36 smithi067 bash[24488]: cluster 2024-02-10T08:39:35.502541+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-10T08:39:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:37 smithi067 bash[17385]: cluster 2024-02-10T08:39:36.260091+0000 mgr.y (mgr.14547) 663 : cluster [DBG] pgmap v1063: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:37 smithi067 bash[17385]: cluster 2024-02-10T08:39:36.523638+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-10T08:39:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:37 smithi067 bash[24488]: cluster 2024-02-10T08:39:36.260091+0000 mgr.y (mgr.14547) 663 : cluster [DBG] pgmap v1063: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:37 smithi067 bash[24488]: cluster 2024-02-10T08:39:36.523638+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-10T08:39:38.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:37 smithi134 bash[18722]: cluster 2024-02-10T08:39:36.260091+0000 mgr.y (mgr.14547) 663 : cluster [DBG] pgmap v1063: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:38.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:37 smithi134 bash[18722]: cluster 2024-02-10T08:39:36.523638+0000 mon.a (mon.0) 2861 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-10T08:39:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:39:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:37] "GET /metrics HTTP/1.1" 200 188495 "" "Prometheus/2.33.4" 2024-02-10T08:39:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:38 smithi067 bash[17385]: cluster 2024-02-10T08:39:37.531415+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-10T08:39:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:38 smithi067 bash[17385]: audit 2024-02-10T08:39:37.599784+0000 mon.c (mon.2) 943 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:38 smithi067 bash[17385]: audit 2024-02-10T08:39:37.600325+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:38 smithi067 bash[24488]: cluster 2024-02-10T08:39:37.531415+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-10T08:39:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:38 smithi067 bash[24488]: audit 2024-02-10T08:39:37.599784+0000 mon.c (mon.2) 943 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:38 smithi067 bash[24488]: audit 2024-02-10T08:39:37.600325+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:39.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:38 smithi134 bash[18722]: cluster 2024-02-10T08:39:37.531415+0000 mon.a (mon.0) 2862 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-10T08:39:39.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:38 smithi134 bash[18722]: audit 2024-02-10T08:39:37.599784+0000 mon.c (mon.2) 943 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:39.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:38 smithi134 bash[18722]: audit 2024-02-10T08:39:37.600325+0000 mon.a (mon.0) 2863 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[17385]: cluster 2024-02-10T08:39:38.260840+0000 mgr.y (mgr.14547) 664 : cluster [DBG] pgmap v1066: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:39:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[17385]: audit 2024-02-10T08:39:38.550364+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[17385]: cluster 2024-02-10T08:39:38.550478+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-10T08:39:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[17385]: audit 2024-02-10T08:39:38.561871+0000 mon.c (mon.2) 944 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[17385]: audit 2024-02-10T08:39:38.562318+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[24488]: cluster 2024-02-10T08:39:38.260840+0000 mgr.y (mgr.14547) 664 : cluster [DBG] pgmap v1066: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:39:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[24488]: audit 2024-02-10T08:39:38.550364+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[24488]: cluster 2024-02-10T08:39:38.550478+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-10T08:39:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[24488]: audit 2024-02-10T08:39:38.561871+0000 mon.c (mon.2) 944 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:39 smithi067 bash[24488]: audit 2024-02-10T08:39:38.562318+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:40.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:39 smithi134 bash[18722]: cluster 2024-02-10T08:39:38.260840+0000 mgr.y (mgr.14547) 664 : cluster [DBG] pgmap v1066: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:39:40.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:39 smithi134 bash[18722]: audit 2024-02-10T08:39:38.550364+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:39 smithi134 bash[18722]: cluster 2024-02-10T08:39:38.550478+0000 mon.a (mon.0) 2865 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-10T08:39:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:39 smithi134 bash[18722]: audit 2024-02-10T08:39:38.561871+0000 mon.c (mon.2) 944 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:39 smithi134 bash[18722]: audit 2024-02-10T08:39:38.562318+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:40.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:40 smithi067 bash[17385]: audit 2024-02-10T08:39:39.557251+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:40.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:40 smithi067 bash[17385]: cluster 2024-02-10T08:39:39.557404+0000 mon.a (mon.0) 2868 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-10T08:39:40.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:40 smithi067 bash[24488]: audit 2024-02-10T08:39:39.557251+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:40.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:40 smithi067 bash[24488]: cluster 2024-02-10T08:39:39.557404+0000 mon.a (mon.0) 2868 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-10T08:39:41.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:40 smithi134 bash[18722]: audit 2024-02-10T08:39:39.557251+0000 mon.a (mon.0) 2867 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:41.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:40 smithi134 bash[18722]: cluster 2024-02-10T08:39:39.557404+0000 mon.a (mon.0) 2868 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-10T08:39:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[17385]: cluster 2024-02-10T08:39:40.261709+0000 mgr.y (mgr.14547) 665 : cluster [DBG] pgmap v1069: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:39:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[17385]: cluster 2024-02-10T08:39:40.569149+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-10T08:39:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[17385]: audit 2024-02-10T08:39:40.614077+0000 mon.c (mon.2) 945 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[17385]: audit 2024-02-10T08:39:40.614713+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[24488]: cluster 2024-02-10T08:39:40.261709+0000 mgr.y (mgr.14547) 665 : cluster [DBG] pgmap v1069: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:39:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[24488]: cluster 2024-02-10T08:39:40.569149+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-10T08:39:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[24488]: audit 2024-02-10T08:39:40.614077+0000 mon.c (mon.2) 945 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:41 smithi067 bash[24488]: audit 2024-02-10T08:39:40.614713+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:41 smithi134 bash[18722]: cluster 2024-02-10T08:39:40.261709+0000 mgr.y (mgr.14547) 665 : cluster [DBG] pgmap v1069: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:39:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:41 smithi134 bash[18722]: cluster 2024-02-10T08:39:40.569149+0000 mon.a (mon.0) 2869 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-10T08:39:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:41 smithi134 bash[18722]: audit 2024-02-10T08:39:40.614077+0000 mon.c (mon.2) 945 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:41 smithi134 bash[18722]: audit 2024-02-10T08:39:40.614713+0000 mon.a (mon.0) 2870 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:42.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:39:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:39:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:42 smithi134 bash[18722]: audit 2024-02-10T08:39:41.589382+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:42 smithi134 bash[18722]: cluster 2024-02-10T08:39:41.589518+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-10T08:39:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:42 smithi134 bash[18722]: audit 2024-02-10T08:39:41.591172+0000 mon.c (mon.2) 946 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:42 smithi134 bash[18722]: audit 2024-02-10T08:39:41.591861+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[17385]: audit 2024-02-10T08:39:41.589382+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[17385]: cluster 2024-02-10T08:39:41.589518+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-10T08:39:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[17385]: audit 2024-02-10T08:39:41.591172+0000 mon.c (mon.2) 946 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[17385]: audit 2024-02-10T08:39:41.591861+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:42.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[24488]: audit 2024-02-10T08:39:41.589382+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[24488]: cluster 2024-02-10T08:39:41.589518+0000 mon.a (mon.0) 2872 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-10T08:39:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[24488]: audit 2024-02-10T08:39:41.591172+0000 mon.c (mon.2) 946 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:42 smithi067 bash[24488]: audit 2024-02-10T08:39:41.591861+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]: dispatch 2024-02-10T08:39:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:39:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:39:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[24488]: cluster 2024-02-10T08:39:42.263749+0000 mgr.y (mgr.14547) 666 : cluster [DBG] pgmap v1072: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-10T08:39:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[24488]: audit 2024-02-10T08:39:42.437944+0000 mgr.y (mgr.14547) 667 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[24488]: cluster 2024-02-10T08:39:42.586362+0000 mon.a (mon.0) 2874 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[24488]: audit 2024-02-10T08:39:42.597998+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[24488]: cluster 2024-02-10T08:39:42.598340+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-10T08:39:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[17385]: cluster 2024-02-10T08:39:42.263749+0000 mgr.y (mgr.14547) 666 : cluster [DBG] pgmap v1072: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-10T08:39:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[17385]: audit 2024-02-10T08:39:42.437944+0000 mgr.y (mgr.14547) 667 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[17385]: cluster 2024-02-10T08:39:42.586362+0000 mon.a (mon.0) 2874 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[17385]: audit 2024-02-10T08:39:42.597998+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:43 smithi067 bash[17385]: cluster 2024-02-10T08:39:42.598340+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-10T08:39:44.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:43 smithi134 bash[18722]: cluster 2024-02-10T08:39:42.263749+0000 mgr.y (mgr.14547) 666 : cluster [DBG] pgmap v1072: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-10T08:39:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:43 smithi134 bash[18722]: audit 2024-02-10T08:39:42.437944+0000 mgr.y (mgr.14547) 667 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:43 smithi134 bash[18722]: cluster 2024-02-10T08:39:42.586362+0000 mon.a (mon.0) 2874 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:43 smithi134 bash[18722]: audit 2024-02-10T08:39:42.597998+0000 mon.a (mon.0) 2875 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-122"}]': finished 2024-02-10T08:39:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:43 smithi134 bash[18722]: cluster 2024-02-10T08:39:42.598340+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-10T08:39:44.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:44 smithi067 bash[17385]: cluster 2024-02-10T08:39:43.618144+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-10T08:39:44.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:44 smithi067 bash[24488]: cluster 2024-02-10T08:39:43.618144+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-10T08:39:45.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:44 smithi134 bash[18722]: cluster 2024-02-10T08:39:43.618144+0000 mon.a (mon.0) 2877 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-10T08:39:46.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:45 smithi134 bash[18722]: cluster 2024-02-10T08:39:44.264573+0000 mgr.y (mgr.14547) 668 : cluster [DBG] pgmap v1075: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:39:46.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:45 smithi134 bash[18722]: cluster 2024-02-10T08:39:44.634515+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-10T08:39:46.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:45 smithi134 bash[18722]: audit 2024-02-10T08:39:44.636960+0000 mon.c (mon.2) 947 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:46.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:45 smithi134 bash[18722]: audit 2024-02-10T08:39:44.638124+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[17385]: cluster 2024-02-10T08:39:44.264573+0000 mgr.y (mgr.14547) 668 : cluster [DBG] pgmap v1075: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:39:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[17385]: cluster 2024-02-10T08:39:44.634515+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-10T08:39:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[17385]: audit 2024-02-10T08:39:44.636960+0000 mon.c (mon.2) 947 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[17385]: audit 2024-02-10T08:39:44.638124+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[24488]: cluster 2024-02-10T08:39:44.264573+0000 mgr.y (mgr.14547) 668 : cluster [DBG] pgmap v1075: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:39:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[24488]: cluster 2024-02-10T08:39:44.634515+0000 mon.a (mon.0) 2878 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-10T08:39:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[24488]: audit 2024-02-10T08:39:44.636960+0000 mon.c (mon.2) 947 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:46.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:45 smithi067 bash[24488]: audit 2024-02-10T08:39:44.638124+0000 mon.a (mon.0) 2879 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:47.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: audit 2024-02-10T08:39:45.638699+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: cluster 2024-02-10T08:39:45.639037+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: audit 2024-02-10T08:39:45.659954+0000 mon.c (mon.2) 948 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: audit 2024-02-10T08:39:45.660651+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: audit 2024-02-10T08:39:46.641380+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: cluster 2024-02-10T08:39:46.641569+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: audit 2024-02-10T08:39:46.645246+0000 mon.c (mon.2) 949 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:47.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:46 smithi134 bash[18722]: audit 2024-02-10T08:39:46.645894+0000 mon.a (mon.0) 2885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: audit 2024-02-10T08:39:45.638699+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: cluster 2024-02-10T08:39:45.639037+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-10T08:39:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: audit 2024-02-10T08:39:45.659954+0000 mon.c (mon.2) 948 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:47.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: audit 2024-02-10T08:39:45.660651+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: audit 2024-02-10T08:39:46.641380+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: cluster 2024-02-10T08:39:46.641569+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: audit 2024-02-10T08:39:46.645246+0000 mon.c (mon.2) 949 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[17385]: audit 2024-02-10T08:39:46.645894+0000 mon.a (mon.0) 2885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: audit 2024-02-10T08:39:45.638699+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: cluster 2024-02-10T08:39:45.639037+0000 mon.a (mon.0) 2881 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: audit 2024-02-10T08:39:45.659954+0000 mon.c (mon.2) 948 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: audit 2024-02-10T08:39:45.660651+0000 mon.a (mon.0) 2882 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: audit 2024-02-10T08:39:46.641380+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: cluster 2024-02-10T08:39:46.641569+0000 mon.a (mon.0) 2884 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: audit 2024-02-10T08:39:46.645246+0000 mon.c (mon.2) 949 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:47.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:46 smithi067 bash[24488]: audit 2024-02-10T08:39:46.645894+0000 mon.a (mon.0) 2885 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:48.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:47 smithi134 bash[18722]: cluster 2024-02-10T08:39:46.265373+0000 mgr.y (mgr.14547) 669 : cluster [DBG] pgmap v1078: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:48.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:47 smithi134 bash[18722]: audit 2024-02-10T08:39:47.044133+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]': finished 2024-02-10T08:39:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:47 smithi134 bash[18722]: cluster 2024-02-10T08:39:47.044283+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-10T08:39:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:47 smithi134 bash[18722]: audit 2024-02-10T08:39:47.048981+0000 mon.c (mon.2) 950 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]: dispatch 2024-02-10T08:39:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:47 smithi134 bash[18722]: audit 2024-02-10T08:39:47.052677+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]: dispatch 2024-02-10T08:39:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:47] "GET /metrics HTTP/1.1" 200 191035 "" "Prometheus/2.33.4" 2024-02-10T08:39:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[24488]: cluster 2024-02-10T08:39:46.265373+0000 mgr.y (mgr.14547) 669 : cluster [DBG] pgmap v1078: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[24488]: audit 2024-02-10T08:39:47.044133+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]': finished 2024-02-10T08:39:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[24488]: cluster 2024-02-10T08:39:47.044283+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-10T08:39:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[24488]: audit 2024-02-10T08:39:47.048981+0000 mon.c (mon.2) 950 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]: dispatch 2024-02-10T08:39:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[24488]: audit 2024-02-10T08:39:47.052677+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]: dispatch 2024-02-10T08:39:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[17385]: cluster 2024-02-10T08:39:46.265373+0000 mgr.y (mgr.14547) 669 : cluster [DBG] pgmap v1078: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[17385]: audit 2024-02-10T08:39:47.044133+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-124"}]': finished 2024-02-10T08:39:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[17385]: cluster 2024-02-10T08:39:47.044283+0000 mon.a (mon.0) 2887 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-10T08:39:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[17385]: audit 2024-02-10T08:39:47.048981+0000 mon.c (mon.2) 950 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]: dispatch 2024-02-10T08:39:48.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:47 smithi067 bash[17385]: audit 2024-02-10T08:39:47.052677+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]: dispatch 2024-02-10T08:39:49.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:48 smithi134 bash[18722]: cluster 2024-02-10T08:39:48.049544+0000 mon.a (mon.0) 2889 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:49.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:48 smithi134 bash[18722]: audit 2024-02-10T08:39:48.060045+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]': finished 2024-02-10T08:39:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:48 smithi134 bash[18722]: cluster 2024-02-10T08:39:48.060173+0000 mon.a (mon.0) 2891 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-10T08:39:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:48 smithi134 bash[18722]: audit 2024-02-10T08:39:48.132861+0000 mon.c (mon.2) 951 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:49.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:48 smithi134 bash[18722]: audit 2024-02-10T08:39:48.133537+0000 mon.a (mon.0) 2892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[17385]: cluster 2024-02-10T08:39:48.049544+0000 mon.a (mon.0) 2889 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[17385]: audit 2024-02-10T08:39:48.060045+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]': finished 2024-02-10T08:39:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[17385]: cluster 2024-02-10T08:39:48.060173+0000 mon.a (mon.0) 2891 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[17385]: audit 2024-02-10T08:39:48.132861+0000 mon.c (mon.2) 951 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[17385]: audit 2024-02-10T08:39:48.133537+0000 mon.a (mon.0) 2892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[24488]: cluster 2024-02-10T08:39:48.049544+0000 mon.a (mon.0) 2889 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[24488]: audit 2024-02-10T08:39:48.060045+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-124", "mode": "writeback"}]': finished 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[24488]: cluster 2024-02-10T08:39:48.060173+0000 mon.a (mon.0) 2891 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[24488]: audit 2024-02-10T08:39:48.132861+0000 mon.c (mon.2) 951 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:48 smithi067 bash[24488]: audit 2024-02-10T08:39:48.133537+0000 mon.a (mon.0) 2892 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:49 smithi134 bash[18722]: cluster 2024-02-10T08:39:48.266234+0000 mgr.y (mgr.14547) 670 : cluster [DBG] pgmap v1082: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:39:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:49 smithi134 bash[18722]: audit 2024-02-10T08:39:49.060911+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:49 smithi134 bash[18722]: cluster 2024-02-10T08:39:49.061108+0000 mon.a (mon.0) 2894 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-10T08:39:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:49 smithi134 bash[18722]: audit 2024-02-10T08:39:49.063786+0000 mon.c (mon.2) 952 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:49 smithi134 bash[18722]: audit 2024-02-10T08:39:49.064303+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[17385]: cluster 2024-02-10T08:39:48.266234+0000 mgr.y (mgr.14547) 670 : cluster [DBG] pgmap v1082: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:39:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[17385]: audit 2024-02-10T08:39:49.060911+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[17385]: cluster 2024-02-10T08:39:49.061108+0000 mon.a (mon.0) 2894 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-10T08:39:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[17385]: audit 2024-02-10T08:39:49.063786+0000 mon.c (mon.2) 952 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[17385]: audit 2024-02-10T08:39:49.064303+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[24488]: cluster 2024-02-10T08:39:48.266234+0000 mgr.y (mgr.14547) 670 : cluster [DBG] pgmap v1082: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:39:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[24488]: audit 2024-02-10T08:39:49.060911+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[24488]: cluster 2024-02-10T08:39:49.061108+0000 mon.a (mon.0) 2894 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-10T08:39:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[24488]: audit 2024-02-10T08:39:49.063786+0000 mon.c (mon.2) 952 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:49 smithi067 bash[24488]: audit 2024-02-10T08:39:49.064303+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]: dispatch 2024-02-10T08:39:51.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:50 smithi134 bash[18722]: cluster 2024-02-10T08:39:50.054702+0000 mon.a (mon.0) 2896 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:51.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:50 smithi134 bash[18722]: audit 2024-02-10T08:39:50.063203+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]': finished 2024-02-10T08:39:51.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:50 smithi134 bash[18722]: cluster 2024-02-10T08:39:50.063357+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-10T08:39:51.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:50 smithi067 bash[17385]: cluster 2024-02-10T08:39:50.054702+0000 mon.a (mon.0) 2896 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:51.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:50 smithi067 bash[17385]: audit 2024-02-10T08:39:50.063203+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]': finished 2024-02-10T08:39:51.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:50 smithi067 bash[17385]: cluster 2024-02-10T08:39:50.063357+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-10T08:39:51.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:50 smithi067 bash[24488]: cluster 2024-02-10T08:39:50.054702+0000 mon.a (mon.0) 2896 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:50 smithi067 bash[24488]: audit 2024-02-10T08:39:50.063203+0000 mon.a (mon.0) 2897 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-124"}]': finished 2024-02-10T08:39:51.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:50 smithi067 bash[24488]: cluster 2024-02-10T08:39:50.063357+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-10T08:39:52.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:51 smithi134 bash[18722]: cluster 2024-02-10T08:39:50.267113+0000 mgr.y (mgr.14547) 671 : cluster [DBG] pgmap v1085: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:39:52.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:51 smithi134 bash[18722]: cluster 2024-02-10T08:39:51.069011+0000 mon.a (mon.0) 2899 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-10T08:39:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:51 smithi067 bash[17385]: cluster 2024-02-10T08:39:50.267113+0000 mgr.y (mgr.14547) 671 : cluster [DBG] pgmap v1085: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:39:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:51 smithi067 bash[17385]: cluster 2024-02-10T08:39:51.069011+0000 mon.a (mon.0) 2899 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-10T08:39:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:51 smithi067 bash[24488]: cluster 2024-02-10T08:39:50.267113+0000 mgr.y (mgr.14547) 671 : cluster [DBG] pgmap v1085: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-10T08:39:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:51 smithi067 bash[24488]: cluster 2024-02-10T08:39:51.069011+0000 mon.a (mon.0) 2899 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-10T08:39:52.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:39:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:39:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[17385]: cluster 2024-02-10T08:39:52.074090+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-10T08:39:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[17385]: audit 2024-02-10T08:39:52.078238+0000 mon.c (mon.2) 953 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[17385]: audit 2024-02-10T08:39:52.083335+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[17385]: cluster 2024-02-10T08:39:52.267969+0000 mgr.y (mgr.14547) 672 : cluster [DBG] pgmap v1088: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:39:53.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[17385]: audit 2024-02-10T08:39:52.449385+0000 mgr.y (mgr.14547) 673 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[24488]: cluster 2024-02-10T08:39:52.074090+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-10T08:39:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[24488]: audit 2024-02-10T08:39:52.078238+0000 mon.c (mon.2) 953 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[24488]: audit 2024-02-10T08:39:52.083335+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[24488]: cluster 2024-02-10T08:39:52.267969+0000 mgr.y (mgr.14547) 672 : cluster [DBG] pgmap v1088: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:39:53.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:53 smithi067 bash[24488]: audit 2024-02-10T08:39:52.449385+0000 mgr.y (mgr.14547) 673 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:39:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:39:53.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:53 smithi134 bash[18722]: cluster 2024-02-10T08:39:52.074090+0000 mon.a (mon.0) 2900 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-10T08:39:53.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:53 smithi134 bash[18722]: audit 2024-02-10T08:39:52.078238+0000 mon.c (mon.2) 953 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:53 smithi134 bash[18722]: audit 2024-02-10T08:39:52.083335+0000 mon.a (mon.0) 2901 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:39:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:53 smithi134 bash[18722]: cluster 2024-02-10T08:39:52.267969+0000 mgr.y (mgr.14547) 672 : cluster [DBG] pgmap v1088: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:39:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:53 smithi134 bash[18722]: audit 2024-02-10T08:39:52.449385+0000 mgr.y (mgr.14547) 673 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: audit 2024-02-10T08:39:53.079339+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: cluster 2024-02-10T08:39:53.079811+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: audit 2024-02-10T08:39:53.082513+0000 mon.c (mon.2) 954 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: audit 2024-02-10T08:39:53.102030+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: audit 2024-02-10T08:39:54.079147+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: cluster 2024-02-10T08:39:54.079358+0000 mon.a (mon.0) 2906 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: audit 2024-02-10T08:39:54.086242+0000 mon.c (mon.2) 955 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[17385]: audit 2024-02-10T08:39:54.086706+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:54.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: audit 2024-02-10T08:39:53.079339+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: cluster 2024-02-10T08:39:53.079811+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: audit 2024-02-10T08:39:53.082513+0000 mon.c (mon.2) 954 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: audit 2024-02-10T08:39:53.102030+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: audit 2024-02-10T08:39:54.079147+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: cluster 2024-02-10T08:39:54.079358+0000 mon.a (mon.0) 2906 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: audit 2024-02-10T08:39:54.086242+0000 mon.c (mon.2) 955 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:54.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:54 smithi067 bash[24488]: audit 2024-02-10T08:39:54.086706+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:54.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: audit 2024-02-10T08:39:53.079339+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: cluster 2024-02-10T08:39:53.079811+0000 mon.a (mon.0) 2903 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: audit 2024-02-10T08:39:53.082513+0000 mon.c (mon.2) 954 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: audit 2024-02-10T08:39:53.102030+0000 mon.a (mon.0) 2904 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: audit 2024-02-10T08:39:54.079147+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: cluster 2024-02-10T08:39:54.079358+0000 mon.a (mon.0) 2906 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: audit 2024-02-10T08:39:54.086242+0000 mon.c (mon.2) 955 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:54.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:54 smithi134 bash[18722]: audit 2024-02-10T08:39:54.086706+0000 mon.a (mon.0) 2907 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:55.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[17385]: cluster 2024-02-10T08:39:54.268784+0000 mgr.y (mgr.14547) 674 : cluster [DBG] pgmap v1091: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:39:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[17385]: audit 2024-02-10T08:39:55.083114+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]': finished 2024-02-10T08:39:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[17385]: cluster 2024-02-10T08:39:55.083264+0000 mon.a (mon.0) 2909 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-10T08:39:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[17385]: audit 2024-02-10T08:39:55.085417+0000 mon.c (mon.2) 956 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]: dispatch 2024-02-10T08:39:55.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[17385]: audit 2024-02-10T08:39:55.085927+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]: dispatch 2024-02-10T08:39:55.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[24488]: cluster 2024-02-10T08:39:54.268784+0000 mgr.y (mgr.14547) 674 : cluster [DBG] pgmap v1091: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:39:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[24488]: audit 2024-02-10T08:39:55.083114+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]': finished 2024-02-10T08:39:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[24488]: cluster 2024-02-10T08:39:55.083264+0000 mon.a (mon.0) 2909 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-10T08:39:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[24488]: audit 2024-02-10T08:39:55.085417+0000 mon.c (mon.2) 956 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]: dispatch 2024-02-10T08:39:55.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:55 smithi067 bash[24488]: audit 2024-02-10T08:39:55.085927+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]: dispatch 2024-02-10T08:39:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:55 smithi134 bash[18722]: cluster 2024-02-10T08:39:54.268784+0000 mgr.y (mgr.14547) 674 : cluster [DBG] pgmap v1091: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-10T08:39:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:55 smithi134 bash[18722]: audit 2024-02-10T08:39:55.083114+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-126"}]': finished 2024-02-10T08:39:55.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:55 smithi134 bash[18722]: cluster 2024-02-10T08:39:55.083264+0000 mon.a (mon.0) 2909 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-10T08:39:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:55 smithi134 bash[18722]: audit 2024-02-10T08:39:55.085417+0000 mon.c (mon.2) 956 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]: dispatch 2024-02-10T08:39:55.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:55 smithi134 bash[18722]: audit 2024-02-10T08:39:55.085927+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]: dispatch 2024-02-10T08:39:56.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: audit 2024-02-10T08:39:55.265770+0000 mon.c (mon.2) 957 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:39:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: audit 2024-02-10T08:39:55.569764+0000 mon.c (mon.2) 958 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:39:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: audit 2024-02-10T08:39:55.571587+0000 mon.c (mon.2) 959 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:39:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: audit 2024-02-10T08:39:55.580314+0000 mon.a (mon.0) 2911 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: cluster 2024-02-10T08:39:56.077801+0000 mon.a (mon.0) 2912 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:56.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: audit 2024-02-10T08:39:56.087724+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]': finished 2024-02-10T08:39:56.372 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[17385]: cluster 2024-02-10T08:39:56.087945+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-10T08:39:56.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: audit 2024-02-10T08:39:55.265770+0000 mon.c (mon.2) 957 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:39:56.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: audit 2024-02-10T08:39:55.569764+0000 mon.c (mon.2) 958 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:39:56.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: audit 2024-02-10T08:39:55.571587+0000 mon.c (mon.2) 959 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:39:56.372 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: audit 2024-02-10T08:39:55.580314+0000 mon.a (mon.0) 2911 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:56.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: cluster 2024-02-10T08:39:56.077801+0000 mon.a (mon.0) 2912 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:56.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: audit 2024-02-10T08:39:56.087724+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]': finished 2024-02-10T08:39:56.373 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:56 smithi067 bash[24488]: cluster 2024-02-10T08:39:56.087945+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-10T08:39:56.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: audit 2024-02-10T08:39:55.265770+0000 mon.c (mon.2) 957 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:39:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: audit 2024-02-10T08:39:55.569764+0000 mon.c (mon.2) 958 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:39:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: audit 2024-02-10T08:39:55.571587+0000 mon.c (mon.2) 959 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:39:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: audit 2024-02-10T08:39:55.580314+0000 mon.a (mon.0) 2911 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:39:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: cluster 2024-02-10T08:39:56.077801+0000 mon.a (mon.0) 2912 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:39:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: audit 2024-02-10T08:39:56.087724+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-126", "mode": "writeback"}]': finished 2024-02-10T08:39:56.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:56 smithi134 bash[18722]: cluster 2024-02-10T08:39:56.087945+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-10T08:39:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[17385]: audit 2024-02-10T08:39:56.183473+0000 mon.c (mon.2) 960 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[17385]: audit 2024-02-10T08:39:56.184053+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:57.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[17385]: cluster 2024-02-10T08:39:56.269613+0000 mgr.y (mgr.14547) 675 : cluster [DBG] pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[24488]: audit 2024-02-10T08:39:56.183473+0000 mon.c (mon.2) 960 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[24488]: audit 2024-02-10T08:39:56.184053+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:57.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[24488]: cluster 2024-02-10T08:39:56.269613+0000 mgr.y (mgr.14547) 675 : cluster [DBG] pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:57 smithi134 bash[18722]: audit 2024-02-10T08:39:56.183473+0000 mon.c (mon.2) 960 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:57 smithi134 bash[18722]: audit 2024-02-10T08:39:56.184053+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:39:57.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:57 smithi134 bash[18722]: cluster 2024-02-10T08:39:56.269613+0000 mgr.y (mgr.14547) 675 : cluster [DBG] pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:39:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:39:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:39:57] "GET /metrics HTTP/1.1" 200 191035 "" "Prometheus/2.33.4" 2024-02-10T08:39:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:58 smithi134 bash[18722]: audit 2024-02-10T08:39:57.116027+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:58 smithi134 bash[18722]: cluster 2024-02-10T08:39:57.116177+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-10T08:39:58.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:58 smithi134 bash[18722]: audit 2024-02-10T08:39:57.117212+0000 mon.c (mon.2) 961 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:58.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:58 smithi134 bash[18722]: audit 2024-02-10T08:39:57.118553+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[17385]: audit 2024-02-10T08:39:57.116027+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[17385]: cluster 2024-02-10T08:39:57.116177+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-10T08:39:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[17385]: audit 2024-02-10T08:39:57.117212+0000 mon.c (mon.2) 961 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:58.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[17385]: audit 2024-02-10T08:39:57.118553+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[24488]: audit 2024-02-10T08:39:57.116027+0000 mon.a (mon.0) 2916 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:39:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[24488]: cluster 2024-02-10T08:39:57.116177+0000 mon.a (mon.0) 2917 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-10T08:39:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[24488]: audit 2024-02-10T08:39:57.117212+0000 mon.c (mon.2) 961 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:58.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:58 smithi067 bash[24488]: audit 2024-02-10T08:39:57.118553+0000 mon.a (mon.0) 2918 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]: dispatch 2024-02-10T08:39:59.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:59 smithi134 bash[18722]: cluster 2024-02-10T08:39:58.111806+0000 mon.a (mon.0) 2919 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:59 smithi134 bash[18722]: audit 2024-02-10T08:39:58.122075+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]': finished 2024-02-10T08:39:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:59 smithi134 bash[18722]: cluster 2024-02-10T08:39:58.122183+0000 mon.a (mon.0) 2921 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-10T08:39:59.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:39:59 smithi134 bash[18722]: cluster 2024-02-10T08:39:58.270450+0000 mgr.y (mgr.14547) 676 : cluster [DBG] pgmap v1097: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-10T08:39:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[17385]: cluster 2024-02-10T08:39:58.111806+0000 mon.a (mon.0) 2919 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[17385]: audit 2024-02-10T08:39:58.122075+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]': finished 2024-02-10T08:39:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[17385]: cluster 2024-02-10T08:39:58.122183+0000 mon.a (mon.0) 2921 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-10T08:39:59.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[17385]: cluster 2024-02-10T08:39:58.270450+0000 mgr.y (mgr.14547) 676 : cluster [DBG] pgmap v1097: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-10T08:39:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[24488]: cluster 2024-02-10T08:39:58.111806+0000 mon.a (mon.0) 2919 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:39:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[24488]: audit 2024-02-10T08:39:58.122075+0000 mon.a (mon.0) 2920 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-126"}]': finished 2024-02-10T08:39:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[24488]: cluster 2024-02-10T08:39:58.122183+0000 mon.a (mon.0) 2921 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-10T08:39:59.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:39:59 smithi067 bash[24488]: cluster 2024-02-10T08:39:58.270450+0000 mgr.y (mgr.14547) 676 : cluster [DBG] pgmap v1097: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-02-10T08:40:00.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:00 smithi134 bash[18722]: cluster 2024-02-10T08:39:59.138511+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-10T08:40:00.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:00 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.000198+0000 mon.a (mon.0) 2923 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-10T08:40:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:00 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.000262+0000 mon.a (mon.0) 2924 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-10T08:40:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:00 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.000294+0000 mon.a (mon.0) 2925 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T08:40:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:00 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.000309+0000 mon.a (mon.0) 2926 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi067-54339-101' 2024-02-10T08:40:00.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:00 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.000339+0000 mon.a (mon.0) 2927 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T08:40:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[17385]: cluster 2024-02-10T08:39:59.138511+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-10T08:40:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.000198+0000 mon.a (mon.0) 2923 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-10T08:40:00.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.000262+0000 mon.a (mon.0) 2924 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.000294+0000 mon.a (mon.0) 2925 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.000309+0000 mon.a (mon.0) 2926 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi067-54339-101' 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.000339+0000 mon.a (mon.0) 2927 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[24488]: cluster 2024-02-10T08:39:59.138511+0000 mon.a (mon.0) 2922 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.000198+0000 mon.a (mon.0) 2923 : cluster [WRN] Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.000262+0000 mon.a (mon.0) 2924 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.000294+0000 mon.a (mon.0) 2925 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.000309+0000 mon.a (mon.0) 2926 : cluster [WRN] application not enabled on pool 'test-rados-api-smithi067-54339-101' 2024-02-10T08:40:00.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:00 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.000339+0000 mon.a (mon.0) 2927 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-10T08:40:01.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:01 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.147235+0000 mon.a (mon.0) 2928 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-10T08:40:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:01 smithi134 bash[18722]: audit 2024-02-10T08:40:00.150189+0000 mon.c (mon.2) 962 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:01 smithi134 bash[18722]: audit 2024-02-10T08:40:00.159852+0000 mon.a (mon.0) 2929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:01 smithi134 bash[18722]: cluster 2024-02-10T08:40:00.271356+0000 mgr.y (mgr.14547) 677 : cluster [DBG] pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:40:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:01 smithi134 bash[18722]: audit 2024-02-10T08:40:01.149027+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:01.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:01 smithi134 bash[18722]: cluster 2024-02-10T08:40:01.149172+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-10T08:40:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.147235+0000 mon.a (mon.0) 2928 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-10T08:40:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[17385]: audit 2024-02-10T08:40:00.150189+0000 mon.c (mon.2) 962 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[17385]: audit 2024-02-10T08:40:00.159852+0000 mon.a (mon.0) 2929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[17385]: cluster 2024-02-10T08:40:00.271356+0000 mgr.y (mgr.14547) 677 : cluster [DBG] pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:40:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[17385]: audit 2024-02-10T08:40:01.149027+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:01.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[17385]: cluster 2024-02-10T08:40:01.149172+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-10T08:40:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.147235+0000 mon.a (mon.0) 2928 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-10T08:40:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[24488]: audit 2024-02-10T08:40:00.150189+0000 mon.c (mon.2) 962 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[24488]: audit 2024-02-10T08:40:00.159852+0000 mon.a (mon.0) 2929 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[24488]: cluster 2024-02-10T08:40:00.271356+0000 mgr.y (mgr.14547) 677 : cluster [DBG] pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-10T08:40:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[24488]: audit 2024-02-10T08:40:01.149027+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:01.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:01 smithi067 bash[24488]: cluster 2024-02-10T08:40:01.149172+0000 mon.a (mon.0) 2931 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-10T08:40:02.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:02 smithi134 bash[18722]: audit 2024-02-10T08:40:01.155891+0000 mon.c (mon.2) 963 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:02.462 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:02 smithi134 bash[18722]: audit 2024-02-10T08:40:01.166164+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:02.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:02 smithi067 bash[17385]: audit 2024-02-10T08:40:01.155891+0000 mon.c (mon.2) 963 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:02.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:02 smithi067 bash[17385]: audit 2024-02-10T08:40:01.166164+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:02.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:02 smithi067 bash[24488]: audit 2024-02-10T08:40:01.155891+0000 mon.c (mon.2) 963 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:02.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:02 smithi067 bash[24488]: audit 2024-02-10T08:40:01.166164+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:02.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:40:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:40:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:40:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[18722]: audit 2024-02-10T08:40:02.173390+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[18722]: cluster 2024-02-10T08:40:02.173530+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-10T08:40:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[18722]: audit 2024-02-10T08:40:02.175346+0000 mon.c (mon.2) 964 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[18722]: audit 2024-02-10T08:40:02.176797+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[18722]: cluster 2024-02-10T08:40:02.272169+0000 mgr.y (mgr.14547) 678 : cluster [DBG] pgmap v1103: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:03.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:03 smithi134 bash[18722]: audit 2024-02-10T08:40:02.460861+0000 mgr.y (mgr.14547) 679 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[17385]: audit 2024-02-10T08:40:02.173390+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[17385]: cluster 2024-02-10T08:40:02.173530+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-10T08:40:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[17385]: audit 2024-02-10T08:40:02.175346+0000 mon.c (mon.2) 964 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:03.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[17385]: audit 2024-02-10T08:40:02.176797+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[17385]: cluster 2024-02-10T08:40:02.272169+0000 mgr.y (mgr.14547) 678 : cluster [DBG] pgmap v1103: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[17385]: audit 2024-02-10T08:40:02.460861+0000 mgr.y (mgr.14547) 679 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[24488]: audit 2024-02-10T08:40:02.173390+0000 mon.a (mon.0) 2933 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[24488]: cluster 2024-02-10T08:40:02.173530+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[24488]: audit 2024-02-10T08:40:02.175346+0000 mon.c (mon.2) 964 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[24488]: audit 2024-02-10T08:40:02.176797+0000 mon.a (mon.0) 2935 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[24488]: cluster 2024-02-10T08:40:02.272169+0000 mgr.y (mgr.14547) 678 : cluster [DBG] pgmap v1103: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:03.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:03 smithi067 bash[24488]: audit 2024-02-10T08:40:02.460861+0000 mgr.y (mgr.14547) 679 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:04.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:04 smithi134 bash[18722]: audit 2024-02-10T08:40:03.177300+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:40:04.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:04 smithi134 bash[18722]: cluster 2024-02-10T08:40:03.177461+0000 mon.a (mon.0) 2937 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-10T08:40:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:04 smithi134 bash[18722]: audit 2024-02-10T08:40:03.180203+0000 mon.c (mon.2) 965 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:04.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:04 smithi134 bash[18722]: audit 2024-02-10T08:40:03.187660+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[17385]: audit 2024-02-10T08:40:03.177300+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:40:04.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[17385]: cluster 2024-02-10T08:40:03.177461+0000 mon.a (mon.0) 2937 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-10T08:40:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[17385]: audit 2024-02-10T08:40:03.180203+0000 mon.c (mon.2) 965 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:04.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[17385]: audit 2024-02-10T08:40:03.187660+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[24488]: audit 2024-02-10T08:40:03.177300+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:40:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[24488]: cluster 2024-02-10T08:40:03.177461+0000 mon.a (mon.0) 2937 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-10T08:40:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[24488]: audit 2024-02-10T08:40:03.180203+0000 mon.c (mon.2) 965 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:04.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:04 smithi067 bash[24488]: audit 2024-02-10T08:40:03.187660+0000 mon.a (mon.0) 2938 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:05.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:05 smithi134 bash[18722]: audit 2024-02-10T08:40:04.191792+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:40:05.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:05 smithi134 bash[18722]: cluster 2024-02-10T08:40:04.191943+0000 mon.a (mon.0) 2940 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-10T08:40:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:05 smithi134 bash[18722]: audit 2024-02-10T08:40:04.193502+0000 mon.c (mon.2) 966 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:40:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:05 smithi134 bash[18722]: audit 2024-02-10T08:40:04.201840+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:40:05.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:05 smithi134 bash[18722]: cluster 2024-02-10T08:40:04.272993+0000 mgr.y (mgr.14547) 680 : cluster [DBG] pgmap v1106: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[24488]: audit 2024-02-10T08:40:04.191792+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:40:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[24488]: cluster 2024-02-10T08:40:04.191943+0000 mon.a (mon.0) 2940 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-10T08:40:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[24488]: audit 2024-02-10T08:40:04.193502+0000 mon.c (mon.2) 966 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:40:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[24488]: audit 2024-02-10T08:40:04.201840+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:40:05.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[24488]: cluster 2024-02-10T08:40:04.272993+0000 mgr.y (mgr.14547) 680 : cluster [DBG] pgmap v1106: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[17385]: audit 2024-02-10T08:40:04.191792+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:40:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[17385]: cluster 2024-02-10T08:40:04.191943+0000 mon.a (mon.0) 2940 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-10T08:40:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[17385]: audit 2024-02-10T08:40:04.193502+0000 mon.c (mon.2) 966 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:40:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[17385]: audit 2024-02-10T08:40:04.201840+0000 mon.a (mon.0) 2941 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-10T08:40:05.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:05 smithi067 bash[17385]: cluster 2024-02-10T08:40:04.272993+0000 mgr.y (mgr.14547) 680 : cluster [DBG] pgmap v1106: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:06.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:06 smithi134 bash[18722]: audit 2024-02-10T08:40:05.205950+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:40:06.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:06 smithi134 bash[18722]: cluster 2024-02-10T08:40:05.206050+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-10T08:40:06.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:06 smithi067 bash[24488]: audit 2024-02-10T08:40:05.205950+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:40:06.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:06 smithi067 bash[24488]: cluster 2024-02-10T08:40:05.206050+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-10T08:40:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:06 smithi067 bash[17385]: audit 2024-02-10T08:40:05.205950+0000 mon.a (mon.0) 2942 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-10T08:40:06.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:06 smithi067 bash[17385]: cluster 2024-02-10T08:40:05.206050+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-10T08:40:07.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:07 smithi134 bash[18722]: audit 2024-02-10T08:40:06.236957+0000 mon.c (mon.2) 967 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:07.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:07 smithi134 bash[18722]: audit 2024-02-10T08:40:06.237771+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:07.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:07 smithi134 bash[18722]: audit 2024-02-10T08:40:06.238987+0000 mon.c (mon.2) 968 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]: dispatch 2024-02-10T08:40:07.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:07 smithi134 bash[18722]: audit 2024-02-10T08:40:06.239693+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]: dispatch 2024-02-10T08:40:07.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:07 smithi134 bash[18722]: cluster 2024-02-10T08:40:06.273834+0000 mgr.y (mgr.14547) 681 : cluster [DBG] pgmap v1108: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[24488]: audit 2024-02-10T08:40:06.236957+0000 mon.c (mon.2) 967 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[24488]: audit 2024-02-10T08:40:06.237771+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[24488]: audit 2024-02-10T08:40:06.238987+0000 mon.c (mon.2) 968 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]: dispatch 2024-02-10T08:40:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[24488]: audit 2024-02-10T08:40:06.239693+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]: dispatch 2024-02-10T08:40:07.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[24488]: cluster 2024-02-10T08:40:06.273834+0000 mgr.y (mgr.14547) 681 : cluster [DBG] pgmap v1108: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[17385]: audit 2024-02-10T08:40:06.236957+0000 mon.c (mon.2) 967 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[17385]: audit 2024-02-10T08:40:06.237771+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[17385]: audit 2024-02-10T08:40:06.238987+0000 mon.c (mon.2) 968 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]: dispatch 2024-02-10T08:40:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[17385]: audit 2024-02-10T08:40:06.239693+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]: dispatch 2024-02-10T08:40:07.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[17385]: cluster 2024-02-10T08:40:06.273834+0000 mgr.y (mgr.14547) 681 : cluster [DBG] pgmap v1108: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:40:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:07] "GET /metrics HTTP/1.1" 200 191162 "" "Prometheus/2.33.4" 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12113 ms) 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 00000012 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12155 ms) 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: require_osd_release = pacific 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6030 ms) 2024-02-10T08:40:08.243 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP (497690 ms total) 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1023 ms) 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11055 ms) 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18115 ms) 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30196 ms total) 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 2024-02-10T08:40:08.244 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7056 ms) 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8113 ms) 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: waiting for scrub... 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: done waiting 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24542 ms) 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10000 ms) 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7974 ms) 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8101 ms) 2024-02-10T08:40:08.245 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10121 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7816 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12274 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8104 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13173 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7452 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8073 ms) 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-10T08:40:08.246 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-10T08:40:08.247 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: ok, hit_set contains 248:602f83fe:::foo:head 2024-02-10T08:40:08.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:08 smithi134 bash[18722]: audit 2024-02-10T08:40:07.223083+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]': finished 2024-02-10T08:40:08.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:08 smithi134 bash[18722]: cluster 2024-02-10T08:40:07.223199+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-10T08:40:08.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:08 smithi067 bash[24488]: audit 2024-02-10T08:40:07.223083+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]': finished 2024-02-10T08:40:08.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:08 smithi067 bash[24488]: cluster 2024-02-10T08:40:07.223199+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-10T08:40:08.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:08 smithi067 bash[17385]: audit 2024-02-10T08:40:07.223083+0000 mon.a (mon.0) 2946 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-128"}]': finished 2024-02-10T08:40:08.623 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:08 smithi067 bash[17385]: cluster 2024-02-10T08:40:07.223199+0000 mon.a (mon.0) 2947 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-10T08:40:09.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:09 smithi134 bash[18722]: cluster 2024-02-10T08:40:08.241067+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-10T08:40:09.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:09 smithi134 bash[18722]: cluster 2024-02-10T08:40:08.274568+0000 mgr.y (mgr.14547) 682 : cluster [DBG] pgmap v1111: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:09.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:09 smithi067 bash[17385]: cluster 2024-02-10T08:40:08.241067+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-10T08:40:09.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:09 smithi067 bash[17385]: cluster 2024-02-10T08:40:08.274568+0000 mgr.y (mgr.14547) 682 : cluster [DBG] pgmap v1111: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:09.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:09 smithi067 bash[24488]: cluster 2024-02-10T08:40:08.241067+0000 mon.a (mon.0) 2948 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-10T08:40:09.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:09 smithi067 bash[24488]: cluster 2024-02-10T08:40:08.274568+0000 mgr.y (mgr.14547) 682 : cluster [DBG] pgmap v1111: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:10.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:10 smithi134 bash[18722]: cluster 2024-02-10T08:40:09.255696+0000 mon.a (mon.0) 2949 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-10T08:40:10.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:10 smithi134 bash[18722]: audit 2024-02-10T08:40:09.266295+0000 mon.c (mon.2) 969 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:10.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:10 smithi134 bash[18722]: audit 2024-02-10T08:40:09.269778+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:10.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:10 smithi067 bash[24488]: cluster 2024-02-10T08:40:09.255696+0000 mon.a (mon.0) 2949 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-10T08:40:10.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:10 smithi067 bash[24488]: audit 2024-02-10T08:40:09.266295+0000 mon.c (mon.2) 969 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:10.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:10 smithi067 bash[24488]: audit 2024-02-10T08:40:09.269778+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:10.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:10 smithi067 bash[17385]: cluster 2024-02-10T08:40:09.255696+0000 mon.a (mon.0) 2949 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-10T08:40:10.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:10 smithi067 bash[17385]: audit 2024-02-10T08:40:09.266295+0000 mon.c (mon.2) 969 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:10.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:10 smithi067 bash[17385]: audit 2024-02-10T08:40:09.269778+0000 mon.a (mon.0) 2950 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:11 smithi134 bash[18722]: audit 2024-02-10T08:40:10.260038+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:11.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:11 smithi134 bash[18722]: cluster 2024-02-10T08:40:10.260214+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-10T08:40:11.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:11 smithi134 bash[18722]: audit 2024-02-10T08:40:10.273628+0000 mon.c (mon.2) 970 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:11.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:11 smithi134 bash[18722]: audit 2024-02-10T08:40:10.274248+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:11.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:11 smithi134 bash[18722]: cluster 2024-02-10T08:40:10.275243+0000 mgr.y (mgr.14547) 683 : cluster [DBG] pgmap v1114: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[17385]: audit 2024-02-10T08:40:10.260038+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[17385]: cluster 2024-02-10T08:40:10.260214+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-10T08:40:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[17385]: audit 2024-02-10T08:40:10.273628+0000 mon.c (mon.2) 970 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[17385]: audit 2024-02-10T08:40:10.274248+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:11.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[17385]: cluster 2024-02-10T08:40:10.275243+0000 mgr.y (mgr.14547) 683 : cluster [DBG] pgmap v1114: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:11.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[24488]: audit 2024-02-10T08:40:10.260038+0000 mon.a (mon.0) 2951 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:11.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[24488]: cluster 2024-02-10T08:40:10.260214+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-10T08:40:11.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[24488]: audit 2024-02-10T08:40:10.273628+0000 mon.c (mon.2) 970 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:11.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[24488]: audit 2024-02-10T08:40:10.274248+0000 mon.a (mon.0) 2953 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:11.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:11 smithi067 bash[24488]: cluster 2024-02-10T08:40:10.275243+0000 mgr.y (mgr.14547) 683 : cluster [DBG] pgmap v1114: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:12.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[17385]: audit 2024-02-10T08:40:11.279198+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:12.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[17385]: cluster 2024-02-10T08:40:11.279357+0000 mon.a (mon.0) 2955 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-10T08:40:12.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[17385]: audit 2024-02-10T08:40:11.281652+0000 mon.c (mon.2) 971 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:40:12.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[17385]: audit 2024-02-10T08:40:11.282225+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:40:12.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[24488]: audit 2024-02-10T08:40:11.279198+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:12.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[24488]: cluster 2024-02-10T08:40:11.279357+0000 mon.a (mon.0) 2955 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-10T08:40:12.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[24488]: audit 2024-02-10T08:40:11.281652+0000 mon.c (mon.2) 971 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:40:12.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:12 smithi067 bash[24488]: audit 2024-02-10T08:40:11.282225+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:40:12.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:40:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:40:12.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:12 smithi134 bash[18722]: audit 2024-02-10T08:40:11.279198+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:12.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:12 smithi134 bash[18722]: cluster 2024-02-10T08:40:11.279357+0000 mon.a (mon.0) 2955 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-10T08:40:12.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:12 smithi134 bash[18722]: audit 2024-02-10T08:40:11.281652+0000 mon.c (mon.2) 971 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:40:12.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:12 smithi134 bash[18722]: audit 2024-02-10T08:40:11.282225+0000 mon.a (mon.0) 2956 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-10T08:40:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:40:13.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[18722]: cluster 2024-02-10T08:40:12.277552+0000 mgr.y (mgr.14547) 684 : cluster [DBG] pgmap v1116: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[18722]: audit 2024-02-10T08:40:12.284735+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]': finished 2024-02-10T08:40:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[18722]: cluster 2024-02-10T08:40:12.284870+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-10T08:40:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[18722]: audit 2024-02-10T08:40:12.288412+0000 mon.c (mon.2) 972 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:40:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[18722]: audit 2024-02-10T08:40:12.293497+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:40:13.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:13 smithi134 bash[18722]: audit 2024-02-10T08:40:12.470848+0000 mgr.y (mgr.14547) 685 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[17385]: cluster 2024-02-10T08:40:12.277552+0000 mgr.y (mgr.14547) 684 : cluster [DBG] pgmap v1116: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:13.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[17385]: audit 2024-02-10T08:40:12.284735+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]': finished 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[17385]: cluster 2024-02-10T08:40:12.284870+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[17385]: audit 2024-02-10T08:40:12.288412+0000 mon.c (mon.2) 972 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[17385]: audit 2024-02-10T08:40:12.293497+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[17385]: audit 2024-02-10T08:40:12.470848+0000 mgr.y (mgr.14547) 685 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[24488]: cluster 2024-02-10T08:40:12.277552+0000 mgr.y (mgr.14547) 684 : cluster [DBG] pgmap v1116: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[24488]: audit 2024-02-10T08:40:12.284735+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_count","val": "3"}]': finished 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[24488]: cluster 2024-02-10T08:40:12.284870+0000 mon.a (mon.0) 2958 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[24488]: audit 2024-02-10T08:40:12.288412+0000 mon.c (mon.2) 972 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[24488]: audit 2024-02-10T08:40:12.293497+0000 mon.a (mon.0) 2959 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-10T08:40:13.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:13 smithi067 bash[24488]: audit 2024-02-10T08:40:12.470848+0000 mgr.y (mgr.14547) 685 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:14.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[17385]: audit 2024-02-10T08:40:13.308173+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]': finished 2024-02-10T08:40:14.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[17385]: cluster 2024-02-10T08:40:13.308310+0000 mon.a (mon.0) 2961 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-10T08:40:14.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[17385]: audit 2024-02-10T08:40:13.309972+0000 mon.c (mon.2) 973 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:14.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[17385]: audit 2024-02-10T08:40:13.310741+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[24488]: audit 2024-02-10T08:40:13.308173+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]': finished 2024-02-10T08:40:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[24488]: cluster 2024-02-10T08:40:13.308310+0000 mon.a (mon.0) 2961 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-10T08:40:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[24488]: audit 2024-02-10T08:40:13.309972+0000 mon.c (mon.2) 973 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:14.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:14 smithi067 bash[24488]: audit 2024-02-10T08:40:13.310741+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:14.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:14 smithi134 bash[18722]: audit 2024-02-10T08:40:13.308173+0000 mon.a (mon.0) 2960 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_period","val": "3"}]': finished 2024-02-10T08:40:14.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:14 smithi134 bash[18722]: cluster 2024-02-10T08:40:13.308310+0000 mon.a (mon.0) 2961 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-10T08:40:14.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:14 smithi134 bash[18722]: audit 2024-02-10T08:40:13.309972+0000 mon.c (mon.2) 973 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:14.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:14 smithi134 bash[18722]: audit 2024-02-10T08:40:13.310741+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[17385]: cluster 2024-02-10T08:40:14.278591+0000 mgr.y (mgr.14547) 686 : cluster [DBG] pgmap v1119: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[17385]: audit 2024-02-10T08:40:14.311246+0000 mon.a (mon.0) 2963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:40:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[17385]: cluster 2024-02-10T08:40:14.311355+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-10T08:40:15.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[17385]: audit 2024-02-10T08:40:14.314802+0000 mon.c (mon.2) 974 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:40:15.621 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[17385]: audit 2024-02-10T08:40:14.321553+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:40:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[24488]: cluster 2024-02-10T08:40:14.278591+0000 mgr.y (mgr.14547) 686 : cluster [DBG] pgmap v1119: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[24488]: audit 2024-02-10T08:40:14.311246+0000 mon.a (mon.0) 2963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:40:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[24488]: cluster 2024-02-10T08:40:14.311355+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-10T08:40:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[24488]: audit 2024-02-10T08:40:14.314802+0000 mon.c (mon.2) 974 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:40:15.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:15 smithi067 bash[24488]: audit 2024-02-10T08:40:14.321553+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:40:15.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:15 smithi134 bash[18722]: cluster 2024-02-10T08:40:14.278591+0000 mgr.y (mgr.14547) 686 : cluster [DBG] pgmap v1119: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:15.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:15 smithi134 bash[18722]: audit 2024-02-10T08:40:14.311246+0000 mon.a (mon.0) 2963 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:40:15.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:15 smithi134 bash[18722]: cluster 2024-02-10T08:40:14.311355+0000 mon.a (mon.0) 2964 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-10T08:40:15.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:15 smithi134 bash[18722]: audit 2024-02-10T08:40:14.314802+0000 mon.c (mon.2) 974 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:40:15.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:15 smithi134 bash[18722]: audit 2024-02-10T08:40:14.321553+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-10T08:40:16.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:16 smithi067 bash[17385]: audit 2024-02-10T08:40:15.331074+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T08:40:16.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:16 smithi067 bash[17385]: cluster 2024-02-10T08:40:15.331179+0000 mon.a (mon.0) 2967 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-10T08:40:16.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:16 smithi067 bash[24488]: audit 2024-02-10T08:40:15.331074+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T08:40:16.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:16 smithi067 bash[24488]: cluster 2024-02-10T08:40:15.331179+0000 mon.a (mon.0) 2967 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-10T08:40:16.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:16 smithi134 bash[18722]: audit 2024-02-10T08:40:15.331074+0000 mon.a (mon.0) 2966 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-10T08:40:16.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:16 smithi134 bash[18722]: cluster 2024-02-10T08:40:15.331179+0000 mon.a (mon.0) 2967 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-10T08:40:17.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:17 smithi067 bash[17385]: cluster 2024-02-10T08:40:16.279591+0000 mgr.y (mgr.14547) 687 : cluster [DBG] pgmap v1122: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:17.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:17 smithi067 bash[24488]: cluster 2024-02-10T08:40:16.279591+0000 mgr.y (mgr.14547) 687 : cluster [DBG] pgmap v1122: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:17.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:17 smithi134 bash[18722]: cluster 2024-02-10T08:40:16.279591+0000 mgr.y (mgr.14547) 687 : cluster [DBG] pgmap v1122: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:40:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:17] "GET /metrics HTTP/1.1" 200 191286 "" "Prometheus/2.33.4" 2024-02-10T08:40:19.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:19 smithi067 bash[24488]: cluster 2024-02-10T08:40:18.281054+0000 mgr.y (mgr.14547) 688 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-02-10T08:40:19.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:19 smithi067 bash[17385]: cluster 2024-02-10T08:40:18.281054+0000 mgr.y (mgr.14547) 688 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-02-10T08:40:19.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:19 smithi134 bash[18722]: cluster 2024-02-10T08:40:18.281054+0000 mgr.y (mgr.14547) 688 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-02-10T08:40:21.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:21 smithi067 bash[17385]: cluster 2024-02-10T08:40:20.281860+0000 mgr.y (mgr.14547) 689 : cluster [DBG] pgmap v1124: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 733 B/s rd, 2.3 KiB/s wr, 1 op/s 2024-02-10T08:40:21.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:21 smithi067 bash[24488]: cluster 2024-02-10T08:40:20.281860+0000 mgr.y (mgr.14547) 689 : cluster [DBG] pgmap v1124: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 733 B/s rd, 2.3 KiB/s wr, 1 op/s 2024-02-10T08:40:21.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:21 smithi134 bash[18722]: cluster 2024-02-10T08:40:20.281860+0000 mgr.y (mgr.14547) 689 : cluster [DBG] pgmap v1124: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 733 B/s rd, 2.3 KiB/s wr, 1 op/s 2024-02-10T08:40:22.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:40:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:40:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:23 smithi067 bash[17385]: cluster 2024-02-10T08:40:22.283854+0000 mgr.y (mgr.14547) 690 : cluster [DBG] pgmap v1125: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-10T08:40:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:23 smithi067 bash[17385]: audit 2024-02-10T08:40:22.482084+0000 mgr.y (mgr.14547) 691 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:23.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:23 smithi067 bash[24488]: cluster 2024-02-10T08:40:22.283854+0000 mgr.y (mgr.14547) 690 : cluster [DBG] pgmap v1125: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-10T08:40:23.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:23 smithi067 bash[24488]: audit 2024-02-10T08:40:22.482084+0000 mgr.y (mgr.14547) 691 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:23.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:23 smithi134 bash[18722]: cluster 2024-02-10T08:40:22.283854+0000 mgr.y (mgr.14547) 690 : cluster [DBG] pgmap v1125: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-02-10T08:40:23.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:23 smithi134 bash[18722]: audit 2024-02-10T08:40:22.482084+0000 mgr.y (mgr.14547) 691 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:23.527 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:40:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:40:25.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:25 smithi067 bash[17385]: cluster 2024-02-10T08:40:24.284652+0000 mgr.y (mgr.14547) 692 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.8 KiB/s wr, 1 op/s 2024-02-10T08:40:25.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:25 smithi067 bash[24488]: cluster 2024-02-10T08:40:24.284652+0000 mgr.y (mgr.14547) 692 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.8 KiB/s wr, 1 op/s 2024-02-10T08:40:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:25 smithi134 bash[18722]: cluster 2024-02-10T08:40:24.284652+0000 mgr.y (mgr.14547) 692 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.8 KiB/s wr, 1 op/s 2024-02-10T08:40:27.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:27 smithi067 bash[17385]: cluster 2024-02-10T08:40:26.285289+0000 mgr.y (mgr.14547) 693 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 934 B/s rd, 5.3 KiB/s wr, 1 op/s 2024-02-10T08:40:27.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:27 smithi067 bash[24488]: cluster 2024-02-10T08:40:26.285289+0000 mgr.y (mgr.14547) 693 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 934 B/s rd, 5.3 KiB/s wr, 1 op/s 2024-02-10T08:40:27.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:27 smithi134 bash[18722]: cluster 2024-02-10T08:40:26.285289+0000 mgr.y (mgr.14547) 693 : cluster [DBG] pgmap v1127: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 934 B/s rd, 5.3 KiB/s wr, 1 op/s 2024-02-10T08:40:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:40:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:27] "GET /metrics HTTP/1.1" 200 191286 "" "Prometheus/2.33.4" 2024-02-10T08:40:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[17385]: audit 2024-02-10T08:40:27.409849+0000 mon.c (mon.2) 975 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[17385]: audit 2024-02-10T08:40:27.410490+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[17385]: audit 2024-02-10T08:40:27.411523+0000 mon.c (mon.2) 976 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]: dispatch 2024-02-10T08:40:28.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[17385]: audit 2024-02-10T08:40:27.412027+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]: dispatch 2024-02-10T08:40:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[24488]: audit 2024-02-10T08:40:27.409849+0000 mon.c (mon.2) 975 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[24488]: audit 2024-02-10T08:40:27.410490+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[24488]: audit 2024-02-10T08:40:27.411523+0000 mon.c (mon.2) 976 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]: dispatch 2024-02-10T08:40:28.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:28 smithi067 bash[24488]: audit 2024-02-10T08:40:27.412027+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]: dispatch 2024-02-10T08:40:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:28 smithi134 bash[18722]: audit 2024-02-10T08:40:27.409849+0000 mon.c (mon.2) 975 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:28 smithi134 bash[18722]: audit 2024-02-10T08:40:27.410490+0000 mon.a (mon.0) 2968 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:28 smithi134 bash[18722]: audit 2024-02-10T08:40:27.411523+0000 mon.c (mon.2) 976 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]: dispatch 2024-02-10T08:40:28.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:28 smithi134 bash[18722]: audit 2024-02-10T08:40:27.412027+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]: dispatch 2024-02-10T08:40:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:29 smithi067 bash[17385]: cluster 2024-02-10T08:40:28.287383+0000 mgr.y (mgr.14547) 694 : cluster [DBG] pgmap v1128: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-10T08:40:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:29 smithi067 bash[17385]: audit 2024-02-10T08:40:28.359388+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]': finished 2024-02-10T08:40:29.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:29 smithi067 bash[17385]: cluster 2024-02-10T08:40:28.359471+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-10T08:40:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:29 smithi067 bash[24488]: cluster 2024-02-10T08:40:28.287383+0000 mgr.y (mgr.14547) 694 : cluster [DBG] pgmap v1128: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-10T08:40:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:29 smithi067 bash[24488]: audit 2024-02-10T08:40:28.359388+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]': finished 2024-02-10T08:40:29.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:29 smithi067 bash[24488]: cluster 2024-02-10T08:40:28.359471+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-10T08:40:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:29 smithi134 bash[18722]: cluster 2024-02-10T08:40:28.287383+0000 mgr.y (mgr.14547) 694 : cluster [DBG] pgmap v1128: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-02-10T08:40:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:29 smithi134 bash[18722]: audit 2024-02-10T08:40:28.359388+0000 mon.a (mon.0) 2970 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-130"}]': finished 2024-02-10T08:40:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:29 smithi134 bash[18722]: cluster 2024-02-10T08:40:28.359471+0000 mon.a (mon.0) 2971 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-10T08:40:30.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:30 smithi134 bash[18722]: cluster 2024-02-10T08:40:29.379855+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-10T08:40:30.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:30 smithi067 bash[17385]: cluster 2024-02-10T08:40:29.379855+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-10T08:40:30.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:30 smithi067 bash[24488]: cluster 2024-02-10T08:40:29.379855+0000 mon.a (mon.0) 2972 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-10T08:40:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:31 smithi134 bash[18722]: cluster 2024-02-10T08:40:30.288168+0000 mgr.y (mgr.14547) 695 : cluster [DBG] pgmap v1131: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:40:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:31 smithi134 bash[18722]: cluster 2024-02-10T08:40:30.387826+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-10T08:40:31.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:31 smithi134 bash[18722]: audit 2024-02-10T08:40:30.396543+0000 mon.c (mon.2) 977 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:31.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:31 smithi134 bash[18722]: audit 2024-02-10T08:40:30.401658+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:31.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:31 smithi134 bash[18722]: audit 2024-02-10T08:40:31.390299+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:31.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:31 smithi134 bash[18722]: cluster 2024-02-10T08:40:31.390525+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-10T08:40:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[24488]: cluster 2024-02-10T08:40:30.288168+0000 mgr.y (mgr.14547) 695 : cluster [DBG] pgmap v1131: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:40:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[24488]: cluster 2024-02-10T08:40:30.387826+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-10T08:40:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[24488]: audit 2024-02-10T08:40:30.396543+0000 mon.c (mon.2) 977 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[24488]: audit 2024-02-10T08:40:30.401658+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[24488]: audit 2024-02-10T08:40:31.390299+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[24488]: cluster 2024-02-10T08:40:31.390525+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[17385]: cluster 2024-02-10T08:40:30.288168+0000 mgr.y (mgr.14547) 695 : cluster [DBG] pgmap v1131: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[17385]: cluster 2024-02-10T08:40:30.387826+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[17385]: audit 2024-02-10T08:40:30.396543+0000 mon.c (mon.2) 977 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[17385]: audit 2024-02-10T08:40:30.401658+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[17385]: audit 2024-02-10T08:40:31.390299+0000 mon.a (mon.0) 2975 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:31.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:31 smithi067 bash[17385]: cluster 2024-02-10T08:40:31.390525+0000 mon.a (mon.0) 2976 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-10T08:40:32.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:40:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:40:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:32 smithi134 bash[18722]: audit 2024-02-10T08:40:31.455916+0000 mon.c (mon.2) 978 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:32.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:32 smithi134 bash[18722]: audit 2024-02-10T08:40:31.456632+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:32.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:32 smithi067 bash[17385]: audit 2024-02-10T08:40:31.455916+0000 mon.c (mon.2) 978 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:32.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:32 smithi067 bash[17385]: audit 2024-02-10T08:40:31.456632+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:32.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:32 smithi067 bash[24488]: audit 2024-02-10T08:40:31.455916+0000 mon.c (mon.2) 978 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:32.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:32 smithi067 bash[24488]: audit 2024-02-10T08:40:31.456632+0000 mon.a (mon.0) 2977 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:33.436 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:40:33.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[18722]: cluster 2024-02-10T08:40:32.290028+0000 mgr.y (mgr.14547) 696 : cluster [DBG] pgmap v1134: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:40:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[18722]: audit 2024-02-10T08:40:32.411842+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[18722]: cluster 2024-02-10T08:40:32.411947+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-10T08:40:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[18722]: audit 2024-02-10T08:40:32.414918+0000 mon.c (mon.2) 979 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[18722]: audit 2024-02-10T08:40:32.419535+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:33.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:33 smithi134 bash[18722]: audit 2024-02-10T08:40:32.490998+0000 mgr.y (mgr.14547) 697 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[17385]: cluster 2024-02-10T08:40:32.290028+0000 mgr.y (mgr.14547) 696 : cluster [DBG] pgmap v1134: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:40:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[17385]: audit 2024-02-10T08:40:32.411842+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[17385]: cluster 2024-02-10T08:40:32.411947+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-10T08:40:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[17385]: audit 2024-02-10T08:40:32.414918+0000 mon.c (mon.2) 979 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[17385]: audit 2024-02-10T08:40:32.419535+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[17385]: audit 2024-02-10T08:40:32.490998+0000 mgr.y (mgr.14547) 697 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[24488]: cluster 2024-02-10T08:40:32.290028+0000 mgr.y (mgr.14547) 696 : cluster [DBG] pgmap v1134: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[24488]: audit 2024-02-10T08:40:32.411842+0000 mon.a (mon.0) 2978 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[24488]: cluster 2024-02-10T08:40:32.411947+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[24488]: audit 2024-02-10T08:40:32.414918+0000 mon.c (mon.2) 979 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[24488]: audit 2024-02-10T08:40:32.419535+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:33.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:33 smithi067 bash[24488]: audit 2024-02-10T08:40:32.490998+0000 mgr.y (mgr.14547) 697 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:34 smithi134 bash[18722]: audit 2024-02-10T08:40:33.428065+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]': finished 2024-02-10T08:40:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:34 smithi134 bash[18722]: cluster 2024-02-10T08:40:33.428228+0000 mon.a (mon.0) 2982 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-10T08:40:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:34 smithi134 bash[18722]: audit 2024-02-10T08:40:33.431265+0000 mon.c (mon.2) 980 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]: dispatch 2024-02-10T08:40:34.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:34 smithi134 bash[18722]: audit 2024-02-10T08:40:33.438923+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]: dispatch 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[17385]: audit 2024-02-10T08:40:33.428065+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]': finished 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[17385]: cluster 2024-02-10T08:40:33.428228+0000 mon.a (mon.0) 2982 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[17385]: audit 2024-02-10T08:40:33.431265+0000 mon.c (mon.2) 980 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]: dispatch 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[17385]: audit 2024-02-10T08:40:33.438923+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]: dispatch 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[24488]: audit 2024-02-10T08:40:33.428065+0000 mon.a (mon.0) 2981 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-132"}]': finished 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[24488]: cluster 2024-02-10T08:40:33.428228+0000 mon.a (mon.0) 2982 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-10T08:40:34.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[24488]: audit 2024-02-10T08:40:33.431265+0000 mon.c (mon.2) 980 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]: dispatch 2024-02-10T08:40:34.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:34 smithi067 bash[24488]: audit 2024-02-10T08:40:33.438923+0000 mon.a (mon.0) 2983 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]: dispatch 2024-02-10T08:40:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:35 smithi134 bash[18722]: cluster 2024-02-10T08:40:34.290909+0000 mgr.y (mgr.14547) 698 : cluster [DBG] pgmap v1137: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T08:40:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:35 smithi134 bash[18722]: cluster 2024-02-10T08:40:34.434498+0000 mon.a (mon.0) 2984 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:40:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:35 smithi134 bash[18722]: audit 2024-02-10T08:40:34.443481+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]': finished 2024-02-10T08:40:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:35 smithi134 bash[18722]: cluster 2024-02-10T08:40:34.443601+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-10T08:40:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:35 smithi134 bash[18722]: audit 2024-02-10T08:40:34.447137+0000 mon.c (mon.2) 981 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:35.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:35 smithi134 bash[18722]: audit 2024-02-10T08:40:34.452271+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:35.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[24488]: cluster 2024-02-10T08:40:34.290909+0000 mgr.y (mgr.14547) 698 : cluster [DBG] pgmap v1137: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T08:40:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[24488]: cluster 2024-02-10T08:40:34.434498+0000 mon.a (mon.0) 2984 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:40:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[24488]: audit 2024-02-10T08:40:34.443481+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]': finished 2024-02-10T08:40:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[24488]: cluster 2024-02-10T08:40:34.443601+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-10T08:40:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[24488]: audit 2024-02-10T08:40:34.447137+0000 mon.c (mon.2) 981 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:35.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[24488]: audit 2024-02-10T08:40:34.452271+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:35.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[17385]: cluster 2024-02-10T08:40:34.290909+0000 mgr.y (mgr.14547) 698 : cluster [DBG] pgmap v1137: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-10T08:40:35.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[17385]: cluster 2024-02-10T08:40:34.434498+0000 mon.a (mon.0) 2984 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:40:35.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[17385]: audit 2024-02-10T08:40:34.443481+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-132", "mode": "writeback"}]': finished 2024-02-10T08:40:35.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[17385]: cluster 2024-02-10T08:40:34.443601+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-10T08:40:35.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[17385]: audit 2024-02-10T08:40:34.447137+0000 mon.c (mon.2) 981 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:35.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:35 smithi067 bash[17385]: audit 2024-02-10T08:40:34.452271+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:40:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:36 smithi134 bash[18722]: audit 2024-02-10T08:40:35.455745+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:40:36.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:36 smithi134 bash[18722]: cluster 2024-02-10T08:40:35.455888+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-10T08:40:36.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:36 smithi134 bash[18722]: audit 2024-02-10T08:40:35.459358+0000 mon.c (mon.2) 982 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:36.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:36 smithi134 bash[18722]: audit 2024-02-10T08:40:35.465059+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[17385]: audit 2024-02-10T08:40:35.455745+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:40:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[17385]: cluster 2024-02-10T08:40:35.455888+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-10T08:40:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[17385]: audit 2024-02-10T08:40:35.459358+0000 mon.c (mon.2) 982 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:36.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[17385]: audit 2024-02-10T08:40:35.465059+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:36.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[24488]: audit 2024-02-10T08:40:35.455745+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:40:36.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[24488]: cluster 2024-02-10T08:40:35.455888+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-10T08:40:36.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[24488]: audit 2024-02-10T08:40:35.459358+0000 mon.c (mon.2) 982 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:36.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:36 smithi067 bash[24488]: audit 2024-02-10T08:40:35.465059+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:40:37.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:37 smithi134 bash[18722]: cluster 2024-02-10T08:40:36.291667+0000 mgr.y (mgr.14547) 699 : cluster [DBG] pgmap v1140: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:37.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:37 smithi134 bash[18722]: audit 2024-02-10T08:40:36.468583+0000 mon.a (mon.0) 2991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:40:37.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:37 smithi134 bash[18722]: cluster 2024-02-10T08:40:36.468742+0000 mon.a (mon.0) 2992 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-10T08:40:37.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:37 smithi134 bash[18722]: audit 2024-02-10T08:40:36.470391+0000 mon.c (mon.2) 983 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:37.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:37 smithi134 bash[18722]: audit 2024-02-10T08:40:36.476935+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[17385]: cluster 2024-02-10T08:40:36.291667+0000 mgr.y (mgr.14547) 699 : cluster [DBG] pgmap v1140: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[17385]: audit 2024-02-10T08:40:36.468583+0000 mon.a (mon.0) 2991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:40:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[17385]: cluster 2024-02-10T08:40:36.468742+0000 mon.a (mon.0) 2992 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-10T08:40:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[17385]: audit 2024-02-10T08:40:36.470391+0000 mon.c (mon.2) 983 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:37.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[17385]: audit 2024-02-10T08:40:36.476935+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[24488]: cluster 2024-02-10T08:40:36.291667+0000 mgr.y (mgr.14547) 699 : cluster [DBG] pgmap v1140: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[24488]: audit 2024-02-10T08:40:36.468583+0000 mon.a (mon.0) 2991 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:40:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[24488]: cluster 2024-02-10T08:40:36.468742+0000 mon.a (mon.0) 2992 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-10T08:40:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[24488]: audit 2024-02-10T08:40:36.470391+0000 mon.c (mon.2) 983 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[24488]: audit 2024-02-10T08:40:36.476935+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:40:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:40:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:37] "GET /metrics HTTP/1.1" 200 191407 "" "Prometheus/2.33.4" 2024-02-10T08:40:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:38 smithi134 bash[18722]: cluster 2024-02-10T08:40:37.471592+0000 mon.a (mon.0) 2994 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:40:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:38 smithi134 bash[18722]: audit 2024-02-10T08:40:37.480379+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:40:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:38 smithi134 bash[18722]: cluster 2024-02-10T08:40:37.480556+0000 mon.a (mon.0) 2996 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-10T08:40:38.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:38 smithi134 bash[18722]: audit 2024-02-10T08:40:37.487151+0000 mon.c (mon.2) 984 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:40:38.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:38 smithi134 bash[18722]: audit 2024-02-10T08:40:37.493840+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:40:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[17385]: cluster 2024-02-10T08:40:37.471592+0000 mon.a (mon.0) 2994 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:40:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[17385]: audit 2024-02-10T08:40:37.480379+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:40:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[17385]: cluster 2024-02-10T08:40:37.480556+0000 mon.a (mon.0) 2996 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-10T08:40:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[17385]: audit 2024-02-10T08:40:37.487151+0000 mon.c (mon.2) 984 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:40:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[17385]: audit 2024-02-10T08:40:37.493840+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:40:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[24488]: cluster 2024-02-10T08:40:37.471592+0000 mon.a (mon.0) 2994 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:40:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[24488]: audit 2024-02-10T08:40:37.480379+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:40:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[24488]: cluster 2024-02-10T08:40:37.480556+0000 mon.a (mon.0) 2996 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-10T08:40:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[24488]: audit 2024-02-10T08:40:37.487151+0000 mon.c (mon.2) 984 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:40:38.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:38 smithi067 bash[24488]: audit 2024-02-10T08:40:37.493840+0000 mon.a (mon.0) 2997 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:40:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:39 smithi134 bash[18722]: cluster 2024-02-10T08:40:38.292563+0000 mgr.y (mgr.14547) 700 : cluster [DBG] pgmap v1143: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:39 smithi134 bash[18722]: audit 2024-02-10T08:40:38.499291+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:40:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:39 smithi134 bash[18722]: cluster 2024-02-10T08:40:38.499424+0000 mon.a (mon.0) 2999 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-10T08:40:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:39 smithi134 bash[18722]: audit 2024-02-10T08:40:38.503087+0000 mon.c (mon.2) 985 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:40:39.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:39 smithi134 bash[18722]: audit 2024-02-10T08:40:38.508441+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:40:39.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[24488]: cluster 2024-02-10T08:40:38.292563+0000 mgr.y (mgr.14547) 700 : cluster [DBG] pgmap v1143: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:39.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[24488]: audit 2024-02-10T08:40:38.499291+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[24488]: cluster 2024-02-10T08:40:38.499424+0000 mon.a (mon.0) 2999 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[24488]: audit 2024-02-10T08:40:38.503087+0000 mon.c (mon.2) 985 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[24488]: audit 2024-02-10T08:40:38.508441+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[17385]: cluster 2024-02-10T08:40:38.292563+0000 mgr.y (mgr.14547) 700 : cluster [DBG] pgmap v1143: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[17385]: audit 2024-02-10T08:40:38.499291+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[17385]: cluster 2024-02-10T08:40:38.499424+0000 mon.a (mon.0) 2999 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[17385]: audit 2024-02-10T08:40:38.503087+0000 mon.c (mon.2) 985 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:40:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:39 smithi067 bash[17385]: audit 2024-02-10T08:40:38.508441+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-10T08:40:40.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[17385]: audit 2024-02-10T08:40:39.512873+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T08:40:40.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[17385]: cluster 2024-02-10T08:40:39.513039+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-10T08:40:40.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[17385]: audit 2024-02-10T08:40:39.517102+0000 mon.c (mon.2) 986 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:40:40.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[17385]: audit 2024-02-10T08:40:39.525023+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:40:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[24488]: audit 2024-02-10T08:40:39.512873+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T08:40:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[24488]: cluster 2024-02-10T08:40:39.513039+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-10T08:40:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[24488]: audit 2024-02-10T08:40:39.517102+0000 mon.c (mon.2) 986 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:40:40.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:40 smithi067 bash[24488]: audit 2024-02-10T08:40:39.525023+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:40:41.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:40 smithi134 bash[18722]: audit 2024-02-10T08:40:39.512873+0000 mon.a (mon.0) 3001 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-10T08:40:41.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:40 smithi134 bash[18722]: cluster 2024-02-10T08:40:39.513039+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-10T08:40:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:40 smithi134 bash[18722]: audit 2024-02-10T08:40:39.517102+0000 mon.c (mon.2) 986 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:40:41.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:40 smithi134 bash[18722]: audit 2024-02-10T08:40:39.525023+0000 mon.a (mon.0) 3003 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-10T08:40:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[17385]: cluster 2024-02-10T08:40:40.293421+0000 mgr.y (mgr.14547) 701 : cluster [DBG] pgmap v1146: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[17385]: audit 2024-02-10T08:40:40.532535+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T08:40:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[17385]: cluster 2024-02-10T08:40:40.532784+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-10T08:40:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[17385]: audit 2024-02-10T08:40:40.598235+0000 mon.c (mon.2) 987 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[17385]: audit 2024-02-10T08:40:40.599097+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[24488]: cluster 2024-02-10T08:40:40.293421+0000 mgr.y (mgr.14547) 701 : cluster [DBG] pgmap v1146: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[24488]: audit 2024-02-10T08:40:40.532535+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T08:40:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[24488]: cluster 2024-02-10T08:40:40.532784+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-10T08:40:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[24488]: audit 2024-02-10T08:40:40.598235+0000 mon.c (mon.2) 987 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:41.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:41 smithi067 bash[24488]: audit 2024-02-10T08:40:40.599097+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:41 smithi134 bash[18722]: cluster 2024-02-10T08:40:40.293421+0000 mgr.y (mgr.14547) 701 : cluster [DBG] pgmap v1146: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:40:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:41 smithi134 bash[18722]: audit 2024-02-10T08:40:40.532535+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-10T08:40:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:41 smithi134 bash[18722]: cluster 2024-02-10T08:40:40.532784+0000 mon.a (mon.0) 3005 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-10T08:40:42.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:41 smithi134 bash[18722]: audit 2024-02-10T08:40:40.598235+0000 mon.c (mon.2) 987 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:42.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:41 smithi134 bash[18722]: audit 2024-02-10T08:40:40.599097+0000 mon.a (mon.0) 3006 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:42.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:40:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:40:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:42 smithi134 bash[18722]: audit 2024-02-10T08:40:41.551261+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:40:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:42 smithi134 bash[18722]: cluster 2024-02-10T08:40:41.551383+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-10T08:40:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:42 smithi134 bash[18722]: audit 2024-02-10T08:40:41.554642+0000 mon.c (mon.2) 988 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:42.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:42 smithi134 bash[18722]: audit 2024-02-10T08:40:41.555297+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[17385]: audit 2024-02-10T08:40:41.551261+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:40:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[17385]: cluster 2024-02-10T08:40:41.551383+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-10T08:40:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[17385]: audit 2024-02-10T08:40:41.554642+0000 mon.c (mon.2) 988 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:42.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[17385]: audit 2024-02-10T08:40:41.555297+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[24488]: audit 2024-02-10T08:40:41.551261+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:40:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[24488]: cluster 2024-02-10T08:40:41.551383+0000 mon.a (mon.0) 3008 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-10T08:40:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[24488]: audit 2024-02-10T08:40:41.554642+0000 mon.c (mon.2) 988 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:42.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:42 smithi067 bash[24488]: audit 2024-02-10T08:40:41.555297+0000 mon.a (mon.0) 3009 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: cluster 2024-02-10T08:40:42.295563+0000 mgr.y (mgr.14547) 702 : cluster [DBG] pgmap v1149: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: audit 2024-02-10T08:40:42.495067+0000 mgr.y (mgr.14547) 703 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: audit 2024-02-10T08:40:42.557362+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]': finished 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: cluster 2024-02-10T08:40:42.557550+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: audit 2024-02-10T08:40:42.608232+0000 mon.c (mon.2) 989 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: audit 2024-02-10T08:40:42.609012+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: audit 2024-02-10T08:40:42.610329+0000 mon.c (mon.2) 990 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[24488]: audit 2024-02-10T08:40:42.611057+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: cluster 2024-02-10T08:40:42.295563+0000 mgr.y (mgr.14547) 702 : cluster [DBG] pgmap v1149: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: audit 2024-02-10T08:40:42.495067+0000 mgr.y (mgr.14547) 703 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: audit 2024-02-10T08:40:42.557362+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]': finished 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: cluster 2024-02-10T08:40:42.557550+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: audit 2024-02-10T08:40:42.608232+0000 mon.c (mon.2) 989 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: audit 2024-02-10T08:40:42.609012+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: audit 2024-02-10T08:40:42.610329+0000 mon.c (mon.2) 990 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:43.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:43 smithi067 bash[17385]: audit 2024-02-10T08:40:42.611057+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:44.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: cluster 2024-02-10T08:40:42.295563+0000 mgr.y (mgr.14547) 702 : cluster [DBG] pgmap v1149: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:40:44.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: audit 2024-02-10T08:40:42.495067+0000 mgr.y (mgr.14547) 703 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: audit 2024-02-10T08:40:42.557362+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]': finished 2024-02-10T08:40:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: cluster 2024-02-10T08:40:42.557550+0000 mon.a (mon.0) 3011 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-10T08:40:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: audit 2024-02-10T08:40:42.608232+0000 mon.c (mon.2) 989 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: audit 2024-02-10T08:40:42.609012+0000 mon.a (mon.0) 3012 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: audit 2024-02-10T08:40:42.610329+0000 mon.c (mon.2) 990 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:44.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:43 smithi134 bash[18722]: audit 2024-02-10T08:40:42.611057+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-132"}]: dispatch 2024-02-10T08:40:44.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:44 smithi067 bash[17385]: cluster 2024-02-10T08:40:43.577766+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-10T08:40:44.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:44 smithi067 bash[24488]: cluster 2024-02-10T08:40:43.577766+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-10T08:40:45.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:44 smithi134 bash[18722]: cluster 2024-02-10T08:40:43.577766+0000 mon.a (mon.0) 3014 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-10T08:40:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[17385]: cluster 2024-02-10T08:40:44.296352+0000 mgr.y (mgr.14547) 704 : cluster [DBG] pgmap v1152: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:40:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[17385]: cluster 2024-02-10T08:40:44.590675+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-10T08:40:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[17385]: audit 2024-02-10T08:40:44.593969+0000 mon.c (mon.2) 991 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[17385]: audit 2024-02-10T08:40:44.594898+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[24488]: cluster 2024-02-10T08:40:44.296352+0000 mgr.y (mgr.14547) 704 : cluster [DBG] pgmap v1152: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:40:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[24488]: cluster 2024-02-10T08:40:44.590675+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-10T08:40:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[24488]: audit 2024-02-10T08:40:44.593969+0000 mon.c (mon.2) 991 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:45.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:45 smithi067 bash[24488]: audit 2024-02-10T08:40:44.594898+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:46.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:45 smithi134 bash[18722]: cluster 2024-02-10T08:40:44.296352+0000 mgr.y (mgr.14547) 704 : cluster [DBG] pgmap v1152: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:40:46.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:45 smithi134 bash[18722]: cluster 2024-02-10T08:40:44.590675+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-10T08:40:46.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:45 smithi134 bash[18722]: audit 2024-02-10T08:40:44.593969+0000 mon.c (mon.2) 991 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:46.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:45 smithi134 bash[18722]: audit 2024-02-10T08:40:44.594898+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:40:46.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[17385]: audit 2024-02-10T08:40:45.601367+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:46.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[17385]: cluster 2024-02-10T08:40:45.601523+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-10T08:40:46.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[17385]: audit 2024-02-10T08:40:45.625102+0000 mon.c (mon.2) 992 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:46.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[17385]: audit 2024-02-10T08:40:45.625654+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:46.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[24488]: audit 2024-02-10T08:40:45.601367+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:46.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[24488]: cluster 2024-02-10T08:40:45.601523+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-10T08:40:46.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[24488]: audit 2024-02-10T08:40:45.625102+0000 mon.c (mon.2) 992 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:46.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:46 smithi067 bash[24488]: audit 2024-02-10T08:40:45.625654+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:47.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:46 smithi134 bash[18722]: audit 2024-02-10T08:40:45.601367+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:40:47.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:46 smithi134 bash[18722]: cluster 2024-02-10T08:40:45.601523+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-10T08:40:47.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:46 smithi134 bash[18722]: audit 2024-02-10T08:40:45.625102+0000 mon.c (mon.2) 992 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:47.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:46 smithi134 bash[18722]: audit 2024-02-10T08:40:45.625654+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:40:47.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:47] "GET /metrics HTTP/1.1" 200 191528 "" "Prometheus/2.33.4" 2024-02-10T08:40:47.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[24488]: cluster 2024-02-10T08:40:46.297194+0000 mgr.y (mgr.14547) 705 : cluster [DBG] pgmap v1155: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:47.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[24488]: audit 2024-02-10T08:40:46.607617+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:47.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[24488]: cluster 2024-02-10T08:40:46.607791+0000 mon.a (mon.0) 3021 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[24488]: audit 2024-02-10T08:40:46.611682+0000 mon.c (mon.2) 993 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[24488]: audit 2024-02-10T08:40:46.616990+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[17385]: cluster 2024-02-10T08:40:46.297194+0000 mgr.y (mgr.14547) 705 : cluster [DBG] pgmap v1155: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[17385]: audit 2024-02-10T08:40:46.607617+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[17385]: cluster 2024-02-10T08:40:46.607791+0000 mon.a (mon.0) 3021 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[17385]: audit 2024-02-10T08:40:46.611682+0000 mon.c (mon.2) 993 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:40:47.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:47 smithi067 bash[17385]: audit 2024-02-10T08:40:46.616990+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:40:48.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:47 smithi134 bash[18722]: cluster 2024-02-10T08:40:46.297194+0000 mgr.y (mgr.14547) 705 : cluster [DBG] pgmap v1155: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:40:48.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:47 smithi134 bash[18722]: audit 2024-02-10T08:40:46.607617+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:40:48.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:47 smithi134 bash[18722]: cluster 2024-02-10T08:40:46.607791+0000 mon.a (mon.0) 3021 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-10T08:40:48.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:47 smithi134 bash[18722]: audit 2024-02-10T08:40:46.611682+0000 mon.c (mon.2) 993 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:40:48.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:47 smithi134 bash[18722]: audit 2024-02-10T08:40:46.616990+0000 mon.a (mon.0) 3022 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:40:49.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:48 smithi134 bash[18722]: audit 2024-02-10T08:40:47.626506+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]': finished 2024-02-10T08:40:49.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:48 smithi134 bash[18722]: cluster 2024-02-10T08:40:47.626648+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-10T08:40:49.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:48 smithi134 bash[18722]: audit 2024-02-10T08:40:47.628560+0000 mon.c (mon.2) 994 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]: dispatch 2024-02-10T08:40:49.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:48 smithi134 bash[18722]: audit 2024-02-10T08:40:47.629709+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]: dispatch 2024-02-10T08:40:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[17385]: audit 2024-02-10T08:40:47.626506+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]': finished 2024-02-10T08:40:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[17385]: cluster 2024-02-10T08:40:47.626648+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-10T08:40:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[17385]: audit 2024-02-10T08:40:47.628560+0000 mon.c (mon.2) 994 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]: dispatch 2024-02-10T08:40:49.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[17385]: audit 2024-02-10T08:40:47.629709+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]: dispatch 2024-02-10T08:40:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[24488]: audit 2024-02-10T08:40:47.626506+0000 mon.a (mon.0) 3023 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-134"}]': finished 2024-02-10T08:40:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[24488]: cluster 2024-02-10T08:40:47.626648+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-10T08:40:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[24488]: audit 2024-02-10T08:40:47.628560+0000 mon.c (mon.2) 994 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]: dispatch 2024-02-10T08:40:49.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:48 smithi067 bash[24488]: audit 2024-02-10T08:40:47.629709+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]: dispatch 2024-02-10T08:40:50.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:49 smithi134 bash[18722]: cluster 2024-02-10T08:40:48.298049+0000 mgr.y (mgr.14547) 706 : cluster [DBG] pgmap v1158: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:40:50.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:49 smithi134 bash[18722]: cluster 2024-02-10T08:40:48.623527+0000 mon.a (mon.0) 3026 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:40:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:49 smithi134 bash[18722]: audit 2024-02-10T08:40:48.632386+0000 mon.a (mon.0) 3027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]': finished 2024-02-10T08:40:50.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:49 smithi134 bash[18722]: cluster 2024-02-10T08:40:48.632517+0000 mon.a (mon.0) 3028 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-10T08:40:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[17385]: cluster 2024-02-10T08:40:48.298049+0000 mgr.y (mgr.14547) 706 : cluster [DBG] pgmap v1158: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:40:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[17385]: cluster 2024-02-10T08:40:48.623527+0000 mon.a (mon.0) 3026 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:40:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[17385]: audit 2024-02-10T08:40:48.632386+0000 mon.a (mon.0) 3027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]': finished 2024-02-10T08:40:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[17385]: cluster 2024-02-10T08:40:48.632517+0000 mon.a (mon.0) 3028 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-10T08:40:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[24488]: cluster 2024-02-10T08:40:48.298049+0000 mgr.y (mgr.14547) 706 : cluster [DBG] pgmap v1158: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:40:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[24488]: cluster 2024-02-10T08:40:48.623527+0000 mon.a (mon.0) 3026 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:40:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[24488]: audit 2024-02-10T08:40:48.632386+0000 mon.a (mon.0) 3027 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-134", "mode": "readproxy"}]': finished 2024-02-10T08:40:50.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:49 smithi067 bash[24488]: cluster 2024-02-10T08:40:48.632517+0000 mon.a (mon.0) 3028 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-10T08:40:52.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:51 smithi134 bash[18722]: cluster 2024-02-10T08:40:50.298940+0000 mgr.y (mgr.14547) 707 : cluster [DBG] pgmap v1160: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 218 B/s wr, 1 op/s 2024-02-10T08:40:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:51 smithi067 bash[17385]: cluster 2024-02-10T08:40:50.298940+0000 mgr.y (mgr.14547) 707 : cluster [DBG] pgmap v1160: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 218 B/s wr, 1 op/s 2024-02-10T08:40:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:51 smithi067 bash[24488]: cluster 2024-02-10T08:40:50.298940+0000 mgr.y (mgr.14547) 707 : cluster [DBG] pgmap v1160: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 218 B/s wr, 1 op/s 2024-02-10T08:40:52.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:40:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:40:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:53 smithi067 bash[17385]: cluster 2024-02-10T08:40:52.301046+0000 mgr.y (mgr.14547) 708 : cluster [DBG] pgmap v1161: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-10T08:40:53.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:53 smithi067 bash[17385]: audit 2024-02-10T08:40:52.506577+0000 mgr.y (mgr.14547) 709 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:53.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:53 smithi067 bash[24488]: cluster 2024-02-10T08:40:52.301046+0000 mgr.y (mgr.14547) 708 : cluster [DBG] pgmap v1161: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-10T08:40:53.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:53 smithi067 bash[24488]: audit 2024-02-10T08:40:52.506577+0000 mgr.y (mgr.14547) 709 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:40:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:40:53.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:53 smithi134 bash[18722]: cluster 2024-02-10T08:40:52.301046+0000 mgr.y (mgr.14547) 708 : cluster [DBG] pgmap v1161: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-02-10T08:40:53.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:53 smithi134 bash[18722]: audit 2024-02-10T08:40:52.506577+0000 mgr.y (mgr.14547) 709 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:40:55.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:55 smithi067 bash[17385]: cluster 2024-02-10T08:40:54.301870+0000 mgr.y (mgr.14547) 710 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 133 B/s wr, 1 op/s 2024-02-10T08:40:55.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:55 smithi067 bash[24488]: cluster 2024-02-10T08:40:54.301870+0000 mgr.y (mgr.14547) 710 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 133 B/s wr, 1 op/s 2024-02-10T08:40:55.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:55 smithi134 bash[18722]: cluster 2024-02-10T08:40:54.301870+0000 mgr.y (mgr.14547) 710 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 133 B/s wr, 1 op/s 2024-02-10T08:40:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[17385]: audit 2024-02-10T08:40:55.589068+0000 mon.c (mon.2) 995 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:40:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[17385]: audit 2024-02-10T08:40:55.902709+0000 mon.c (mon.2) 996 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:40:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[17385]: audit 2024-02-10T08:40:55.903701+0000 mon.c (mon.2) 997 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:40:56.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[17385]: audit 2024-02-10T08:40:55.910203+0000 mon.a (mon.0) 3029 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:40:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[24488]: audit 2024-02-10T08:40:55.589068+0000 mon.c (mon.2) 995 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:40:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[24488]: audit 2024-02-10T08:40:55.902709+0000 mon.c (mon.2) 996 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:40:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[24488]: audit 2024-02-10T08:40:55.903701+0000 mon.c (mon.2) 997 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:40:56.621 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:56 smithi067 bash[24488]: audit 2024-02-10T08:40:55.910203+0000 mon.a (mon.0) 3029 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:40:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:56 smithi134 bash[18722]: audit 2024-02-10T08:40:55.589068+0000 mon.c (mon.2) 995 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:40:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:56 smithi134 bash[18722]: audit 2024-02-10T08:40:55.902709+0000 mon.c (mon.2) 996 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:40:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:56 smithi134 bash[18722]: audit 2024-02-10T08:40:55.903701+0000 mon.c (mon.2) 997 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:40:56.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:56 smithi134 bash[18722]: audit 2024-02-10T08:40:55.910203+0000 mon.a (mon.0) 3029 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:40:57.620 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:57 smithi067 bash[17385]: cluster 2024-02-10T08:40:56.302690+0000 mgr.y (mgr.14547) 711 : cluster [DBG] pgmap v1163: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-02-10T08:40:57.620 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:57 smithi067 bash[24488]: cluster 2024-02-10T08:40:56.302690+0000 mgr.y (mgr.14547) 711 : cluster [DBG] pgmap v1163: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-02-10T08:40:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:57 smithi134 bash[18722]: cluster 2024-02-10T08:40:56.302690+0000 mgr.y (mgr.14547) 711 : cluster [DBG] pgmap v1163: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 707 B/s rd, 0 op/s 2024-02-10T08:40:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:40:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:40:57] "GET /metrics HTTP/1.1" 200 191528 "" "Prometheus/2.33.4" 2024-02-10T08:40:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:59 smithi134 bash[18722]: cluster 2024-02-10T08:40:58.304821+0000 mgr.y (mgr.14547) 712 : cluster [DBG] pgmap v1164: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:40:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:59 smithi134 bash[18722]: audit 2024-02-10T08:40:58.892673+0000 mon.c (mon.2) 998 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:59.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:40:59 smithi134 bash[18722]: audit 2024-02-10T08:40:58.893451+0000 mon.a (mon.0) 3030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:59 smithi067 bash[17385]: cluster 2024-02-10T08:40:58.304821+0000 mgr.y (mgr.14547) 712 : cluster [DBG] pgmap v1164: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:40:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:59 smithi067 bash[17385]: audit 2024-02-10T08:40:58.892673+0000 mon.c (mon.2) 998 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:40:59 smithi067 bash[17385]: audit 2024-02-10T08:40:58.893451+0000 mon.a (mon.0) 3030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:59.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:59 smithi067 bash[24488]: cluster 2024-02-10T08:40:58.304821+0000 mgr.y (mgr.14547) 712 : cluster [DBG] pgmap v1164: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:40:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:59 smithi067 bash[24488]: audit 2024-02-10T08:40:58.892673+0000 mon.c (mon.2) 998 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:40:59.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:40:59 smithi067 bash[24488]: audit 2024-02-10T08:40:58.893451+0000 mon.a (mon.0) 3030 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:00 smithi134 bash[18722]: audit 2024-02-10T08:40:59.381702+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:00 smithi134 bash[18722]: cluster 2024-02-10T08:40:59.381844+0000 mon.a (mon.0) 3032 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-10T08:41:00.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:00 smithi134 bash[18722]: audit 2024-02-10T08:40:59.383816+0000 mon.c (mon.2) 999 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:00.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:00 smithi134 bash[18722]: audit 2024-02-10T08:40:59.386053+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[17385]: audit 2024-02-10T08:40:59.381702+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[17385]: cluster 2024-02-10T08:40:59.381844+0000 mon.a (mon.0) 3032 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-10T08:41:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[17385]: audit 2024-02-10T08:40:59.383816+0000 mon.c (mon.2) 999 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:00.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[17385]: audit 2024-02-10T08:40:59.386053+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[24488]: audit 2024-02-10T08:40:59.381702+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[24488]: cluster 2024-02-10T08:40:59.381844+0000 mon.a (mon.0) 3032 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-10T08:41:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[24488]: audit 2024-02-10T08:40:59.383816+0000 mon.c (mon.2) 999 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:00.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:00 smithi067 bash[24488]: audit 2024-02-10T08:40:59.386053+0000 mon.a (mon.0) 3033 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:01.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: cluster 2024-02-10T08:41:00.305692+0000 mgr.y (mgr.14547) 713 : cluster [DBG] pgmap v1166: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: cluster 2024-02-10T08:41:00.377711+0000 mon.a (mon.0) 3034 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: audit 2024-02-10T08:41:00.385640+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]': finished 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: cluster 2024-02-10T08:41:00.385772+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: audit 2024-02-10T08:41:00.429604+0000 mon.c (mon.2) 1000 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: audit 2024-02-10T08:41:00.430285+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: audit 2024-02-10T08:41:00.431317+0000 mon.c (mon.2) 1001 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:01.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:01 smithi134 bash[18722]: audit 2024-02-10T08:41:00.431788+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: cluster 2024-02-10T08:41:00.305692+0000 mgr.y (mgr.14547) 713 : cluster [DBG] pgmap v1166: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:41:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: cluster 2024-02-10T08:41:00.377711+0000 mon.a (mon.0) 3034 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:41:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: audit 2024-02-10T08:41:00.385640+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]': finished 2024-02-10T08:41:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: cluster 2024-02-10T08:41:00.385772+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-10T08:41:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: audit 2024-02-10T08:41:00.429604+0000 mon.c (mon.2) 1000 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:01.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: audit 2024-02-10T08:41:00.430285+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: audit 2024-02-10T08:41:00.431317+0000 mon.c (mon.2) 1001 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[17385]: audit 2024-02-10T08:41:00.431788+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: cluster 2024-02-10T08:41:00.305692+0000 mgr.y (mgr.14547) 713 : cluster [DBG] pgmap v1166: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: cluster 2024-02-10T08:41:00.377711+0000 mon.a (mon.0) 3034 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: audit 2024-02-10T08:41:00.385640+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]': finished 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: cluster 2024-02-10T08:41:00.385772+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: audit 2024-02-10T08:41:00.429604+0000 mon.c (mon.2) 1000 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: audit 2024-02-10T08:41:00.430285+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: audit 2024-02-10T08:41:00.431317+0000 mon.c (mon.2) 1001 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:01.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:01 smithi067 bash[24488]: audit 2024-02-10T08:41:00.431788+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-134"}]: dispatch 2024-02-10T08:41:02.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:02 smithi134 bash[18722]: cluster 2024-02-10T08:41:01.403516+0000 mon.a (mon.0) 3039 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-10T08:41:02.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:41:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:41:02.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:02 smithi067 bash[24488]: cluster 2024-02-10T08:41:01.403516+0000 mon.a (mon.0) 3039 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-10T08:41:02.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:02 smithi067 bash[17385]: cluster 2024-02-10T08:41:01.403516+0000 mon.a (mon.0) 3039 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-10T08:41:03.415 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:41:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:41:03.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:03 smithi134 bash[18722]: cluster 2024-02-10T08:41:02.307618+0000 mgr.y (mgr.14547) 714 : cluster [DBG] pgmap v1169: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:41:03.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:03 smithi134 bash[18722]: cluster 2024-02-10T08:41:02.416836+0000 mon.a (mon.0) 3040 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-10T08:41:03.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:03 smithi134 bash[18722]: audit 2024-02-10T08:41:02.421856+0000 mon.c (mon.2) 1002 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:03.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:03 smithi134 bash[18722]: audit 2024-02-10T08:41:02.423453+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:03.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:03 smithi134 bash[18722]: audit 2024-02-10T08:41:02.515102+0000 mgr.y (mgr.14547) 715 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[17385]: cluster 2024-02-10T08:41:02.307618+0000 mgr.y (mgr.14547) 714 : cluster [DBG] pgmap v1169: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:41:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[17385]: cluster 2024-02-10T08:41:02.416836+0000 mon.a (mon.0) 3040 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-10T08:41:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[17385]: audit 2024-02-10T08:41:02.421856+0000 mon.c (mon.2) 1002 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:03.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[17385]: audit 2024-02-10T08:41:02.423453+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:03.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[17385]: audit 2024-02-10T08:41:02.515102+0000 mgr.y (mgr.14547) 715 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[24488]: cluster 2024-02-10T08:41:02.307618+0000 mgr.y (mgr.14547) 714 : cluster [DBG] pgmap v1169: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-10T08:41:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[24488]: cluster 2024-02-10T08:41:02.416836+0000 mon.a (mon.0) 3040 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-10T08:41:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[24488]: audit 2024-02-10T08:41:02.421856+0000 mon.c (mon.2) 1002 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[24488]: audit 2024-02-10T08:41:02.423453+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:03.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:03 smithi067 bash[24488]: audit 2024-02-10T08:41:02.515102+0000 mgr.y (mgr.14547) 715 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:04.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:04 smithi134 bash[18722]: audit 2024-02-10T08:41:03.425165+0000 mon.a (mon.0) 3042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:04.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:04 smithi134 bash[18722]: cluster 2024-02-10T08:41:03.425342+0000 mon.a (mon.0) 3043 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-10T08:41:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:04 smithi134 bash[18722]: audit 2024-02-10T08:41:03.454476+0000 mon.c (mon.2) 1003 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:04.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:04 smithi134 bash[18722]: audit 2024-02-10T08:41:03.454932+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[24488]: audit 2024-02-10T08:41:03.425165+0000 mon.a (mon.0) 3042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[24488]: cluster 2024-02-10T08:41:03.425342+0000 mon.a (mon.0) 3043 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-10T08:41:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[24488]: audit 2024-02-10T08:41:03.454476+0000 mon.c (mon.2) 1003 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:04.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[24488]: audit 2024-02-10T08:41:03.454932+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:04.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[17385]: audit 2024-02-10T08:41:03.425165+0000 mon.a (mon.0) 3042 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:04.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[17385]: cluster 2024-02-10T08:41:03.425342+0000 mon.a (mon.0) 3043 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-10T08:41:04.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[17385]: audit 2024-02-10T08:41:03.454476+0000 mon.c (mon.2) 1003 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:04.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:04 smithi067 bash[17385]: audit 2024-02-10T08:41:03.454932+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:05.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:05 smithi134 bash[18722]: cluster 2024-02-10T08:41:04.308441+0000 mgr.y (mgr.14547) 716 : cluster [DBG] pgmap v1172: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:05.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:05 smithi134 bash[18722]: audit 2024-02-10T08:41:04.429893+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:41:05.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:05 smithi134 bash[18722]: cluster 2024-02-10T08:41:04.430050+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-10T08:41:05.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:05 smithi134 bash[18722]: audit 2024-02-10T08:41:04.433647+0000 mon.c (mon.2) 1004 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:05.780 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:05 smithi134 bash[18722]: audit 2024-02-10T08:41:04.440733+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[17385]: cluster 2024-02-10T08:41:04.308441+0000 mgr.y (mgr.14547) 716 : cluster [DBG] pgmap v1172: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[17385]: audit 2024-02-10T08:41:04.429893+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:41:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[17385]: cluster 2024-02-10T08:41:04.430050+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-10T08:41:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[17385]: audit 2024-02-10T08:41:04.433647+0000 mon.c (mon.2) 1004 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:05.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[17385]: audit 2024-02-10T08:41:04.440733+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:05.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[24488]: cluster 2024-02-10T08:41:04.308441+0000 mgr.y (mgr.14547) 716 : cluster [DBG] pgmap v1172: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[24488]: audit 2024-02-10T08:41:04.429893+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:41:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[24488]: cluster 2024-02-10T08:41:04.430050+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-10T08:41:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[24488]: audit 2024-02-10T08:41:04.433647+0000 mon.c (mon.2) 1004 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:05.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:05 smithi067 bash[24488]: audit 2024-02-10T08:41:04.440733+0000 mon.a (mon.0) 3047 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:06.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:06 smithi134 bash[18722]: audit 2024-02-10T08:41:05.448814+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]': finished 2024-02-10T08:41:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:06 smithi134 bash[18722]: cluster 2024-02-10T08:41:05.448942+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-10T08:41:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:06 smithi134 bash[18722]: audit 2024-02-10T08:41:05.450176+0000 mon.c (mon.2) 1005 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]: dispatch 2024-02-10T08:41:06.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:06 smithi134 bash[18722]: audit 2024-02-10T08:41:05.450925+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]: dispatch 2024-02-10T08:41:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[17385]: audit 2024-02-10T08:41:05.448814+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]': finished 2024-02-10T08:41:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[17385]: cluster 2024-02-10T08:41:05.448942+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-10T08:41:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[17385]: audit 2024-02-10T08:41:05.450176+0000 mon.c (mon.2) 1005 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]: dispatch 2024-02-10T08:41:06.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[17385]: audit 2024-02-10T08:41:05.450925+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]: dispatch 2024-02-10T08:41:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[24488]: audit 2024-02-10T08:41:05.448814+0000 mon.a (mon.0) 3048 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi067-54339-101", "overlaypool": "test-rados-api-smithi067-54339-136"}]': finished 2024-02-10T08:41:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[24488]: cluster 2024-02-10T08:41:05.448942+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-10T08:41:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[24488]: audit 2024-02-10T08:41:05.450176+0000 mon.c (mon.2) 1005 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]: dispatch 2024-02-10T08:41:06.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:06 smithi067 bash[24488]: audit 2024-02-10T08:41:05.450925+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]: dispatch 2024-02-10T08:41:07.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:07 smithi134 bash[18722]: cluster 2024-02-10T08:41:06.309262+0000 mgr.y (mgr.14547) 717 : cluster [DBG] pgmap v1175: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:07.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:07 smithi134 bash[18722]: cluster 2024-02-10T08:41:06.446198+0000 mon.a (mon.0) 3051 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:41:07.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:07 smithi134 bash[18722]: audit 2024-02-10T08:41:06.457901+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]': finished 2024-02-10T08:41:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:07 smithi134 bash[18722]: cluster 2024-02-10T08:41:06.457997+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-10T08:41:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:07 smithi134 bash[18722]: audit 2024-02-10T08:41:06.518063+0000 mon.c (mon.2) 1006 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:41:07.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:07 smithi134 bash[18722]: audit 2024-02-10T08:41:06.518820+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:41:07.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17385]: cluster 2024-02-10T08:41:06.309262+0000 mgr.y (mgr.14547) 717 : cluster [DBG] pgmap v1175: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:07.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17385]: cluster 2024-02-10T08:41:06.446198+0000 mon.a (mon.0) 3051 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17385]: audit 2024-02-10T08:41:06.457901+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]': finished 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17385]: cluster 2024-02-10T08:41:06.457997+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17385]: audit 2024-02-10T08:41:06.518063+0000 mon.c (mon.2) 1006 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17385]: audit 2024-02-10T08:41:06.518820+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[24488]: cluster 2024-02-10T08:41:06.309262+0000 mgr.y (mgr.14547) 717 : cluster [DBG] pgmap v1175: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[24488]: cluster 2024-02-10T08:41:06.446198+0000 mon.a (mon.0) 3051 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[24488]: audit 2024-02-10T08:41:06.457901+0000 mon.a (mon.0) 3052 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi067-54339-136", "mode": "writeback"}]': finished 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[24488]: cluster 2024-02-10T08:41:06.457997+0000 mon.a (mon.0) 3053 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[24488]: audit 2024-02-10T08:41:06.518063+0000 mon.c (mon.2) 1006 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:41:07.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[24488]: audit 2024-02-10T08:41:06.518820+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-10T08:41:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:41:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:07] "GET /metrics HTTP/1.1" 200 191524 "" "Prometheus/2.33.4" 2024-02-10T08:41:08.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:08 smithi134 bash[18722]: audit 2024-02-10T08:41:07.470522+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:41:08.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:08 smithi134 bash[18722]: cluster 2024-02-10T08:41:07.470695+0000 mon.a (mon.0) 3056 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-10T08:41:08.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:08 smithi134 bash[18722]: audit 2024-02-10T08:41:07.473021+0000 mon.c (mon.2) 1007 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:41:08.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:08 smithi134 bash[18722]: audit 2024-02-10T08:41:07.473687+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:41:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[24488]: audit 2024-02-10T08:41:07.470522+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:41:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[24488]: cluster 2024-02-10T08:41:07.470695+0000 mon.a (mon.0) 3056 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-10T08:41:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[24488]: audit 2024-02-10T08:41:07.473021+0000 mon.c (mon.2) 1007 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:41:08.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[24488]: audit 2024-02-10T08:41:07.473687+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:41:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[17385]: audit 2024-02-10T08:41:07.470522+0000 mon.a (mon.0) 3055 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_count","val": "2"}]': finished 2024-02-10T08:41:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[17385]: cluster 2024-02-10T08:41:07.470695+0000 mon.a (mon.0) 3056 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-10T08:41:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[17385]: audit 2024-02-10T08:41:07.473021+0000 mon.c (mon.2) 1007 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:41:08.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:08 smithi067 bash[17385]: audit 2024-02-10T08:41:07.473687+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-10T08:41:09.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:09 smithi134 bash[18722]: cluster 2024-02-10T08:41:08.310105+0000 mgr.y (mgr.14547) 718 : cluster [DBG] pgmap v1178: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:41:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:09 smithi134 bash[18722]: audit 2024-02-10T08:41:08.480332+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:41:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:09 smithi134 bash[18722]: cluster 2024-02-10T08:41:08.480520+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-10T08:41:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:09 smithi134 bash[18722]: audit 2024-02-10T08:41:08.482692+0000 mon.c (mon.2) 1008 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:41:09.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:09 smithi134 bash[18722]: audit 2024-02-10T08:41:08.483414+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:41:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[17385]: cluster 2024-02-10T08:41:08.310105+0000 mgr.y (mgr.14547) 718 : cluster [DBG] pgmap v1178: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:41:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[17385]: audit 2024-02-10T08:41:08.480332+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:41:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[17385]: cluster 2024-02-10T08:41:08.480520+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-10T08:41:09.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[17385]: audit 2024-02-10T08:41:08.482692+0000 mon.c (mon.2) 1008 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:41:09.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[17385]: audit 2024-02-10T08:41:08.483414+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:41:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[24488]: cluster 2024-02-10T08:41:08.310105+0000 mgr.y (mgr.14547) 718 : cluster [DBG] pgmap v1178: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:41:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[24488]: audit 2024-02-10T08:41:08.480332+0000 mon.a (mon.0) 3058 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_period","val": "600"}]': finished 2024-02-10T08:41:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[24488]: cluster 2024-02-10T08:41:08.480520+0000 mon.a (mon.0) 3059 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-10T08:41:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[24488]: audit 2024-02-10T08:41:08.482692+0000 mon.c (mon.2) 1008 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:41:09.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:09 smithi067 bash[24488]: audit 2024-02-10T08:41:08.483414+0000 mon.a (mon.0) 3060 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-10T08:41:10.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:10 smithi134 bash[18722]: cluster 2024-02-10T08:41:09.476196+0000 mon.a (mon.0) 3061 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:41:10.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:10 smithi134 bash[18722]: audit 2024-02-10T08:41:09.483944+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:41:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:10 smithi134 bash[18722]: cluster 2024-02-10T08:41:09.484139+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-10T08:41:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:10 smithi134 bash[18722]: audit 2024-02-10T08:41:09.489241+0000 mon.c (mon.2) 1009 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:41:10.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:10 smithi134 bash[18722]: audit 2024-02-10T08:41:09.495316+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:41:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[24488]: cluster 2024-02-10T08:41:09.476196+0000 mon.a (mon.0) 3061 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:41:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[24488]: audit 2024-02-10T08:41:09.483944+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:41:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[24488]: cluster 2024-02-10T08:41:09.484139+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-10T08:41:10.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[24488]: audit 2024-02-10T08:41:09.489241+0000 mon.c (mon.2) 1009 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:41:10.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[24488]: audit 2024-02-10T08:41:09.495316+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:41:10.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[17385]: cluster 2024-02-10T08:41:09.476196+0000 mon.a (mon.0) 3061 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-10T08:41:10.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[17385]: audit 2024-02-10T08:41:09.483944+0000 mon.a (mon.0) 3062 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-10T08:41:10.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[17385]: cluster 2024-02-10T08:41:09.484139+0000 mon.a (mon.0) 3063 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-10T08:41:10.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[17385]: audit 2024-02-10T08:41:09.489241+0000 mon.c (mon.2) 1009 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:41:10.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:10 smithi067 bash[17385]: audit 2024-02-10T08:41:09.495316+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-10T08:41:11.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: cluster 2024-02-10T08:41:10.310993+0000 mgr.y (mgr.14547) 719 : cluster [DBG] pgmap v1181: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:41:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: audit 2024-02-10T08:41:10.499144+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:41:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: cluster 2024-02-10T08:41:10.499269+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-10T08:41:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: audit 2024-02-10T08:41:10.500641+0000 mon.c (mon.2) 1010 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:41:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: audit 2024-02-10T08:41:10.502026+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:41:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: audit 2024-02-10T08:41:11.500077+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]': finished 2024-02-10T08:41:11.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:11 smithi134 bash[18722]: cluster 2024-02-10T08:41:11.500275+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-10T08:41:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: cluster 2024-02-10T08:41:10.310993+0000 mgr.y (mgr.14547) 719 : cluster [DBG] pgmap v1181: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:41:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: audit 2024-02-10T08:41:10.499144+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:41:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: cluster 2024-02-10T08:41:10.499269+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-10T08:41:11.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: audit 2024-02-10T08:41:10.500641+0000 mon.c (mon.2) 1010 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: audit 2024-02-10T08:41:10.502026+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: audit 2024-02-10T08:41:11.500077+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]': finished 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[17385]: cluster 2024-02-10T08:41:11.500275+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: cluster 2024-02-10T08:41:10.310993+0000 mgr.y (mgr.14547) 719 : cluster [DBG] pgmap v1181: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: audit 2024-02-10T08:41:10.499144+0000 mon.a (mon.0) 3065 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: cluster 2024-02-10T08:41:10.499269+0000 mon.a (mon.0) 3066 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: audit 2024-02-10T08:41:10.500641+0000 mon.c (mon.2) 1010 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: audit 2024-02-10T08:41:10.502026+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: audit 2024-02-10T08:41:11.500077+0000 mon.a (mon.0) 3068 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-136","var": "target_max_objects","val": "1"}]': finished 2024-02-10T08:41:11.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:11 smithi067 bash[24488]: cluster 2024-02-10T08:41:11.500275+0000 mon.a (mon.0) 3069 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-10T08:41:13.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:41:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:41:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:13 smithi067 bash[17385]: cluster 2024-02-10T08:41:12.313119+0000 mgr.y (mgr.14547) 720 : cluster [DBG] pgmap v1184: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:13.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:13 smithi067 bash[17385]: audit 2024-02-10T08:41:12.526569+0000 mgr.y (mgr.14547) 721 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:13.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:13 smithi067 bash[24488]: cluster 2024-02-10T08:41:12.313119+0000 mgr.y (mgr.14547) 720 : cluster [DBG] pgmap v1184: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:13.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:13 smithi067 bash[24488]: audit 2024-02-10T08:41:12.526569+0000 mgr.y (mgr.14547) 721 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:41:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:41:13.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:13 smithi134 bash[18722]: cluster 2024-02-10T08:41:12.313119+0000 mgr.y (mgr.14547) 720 : cluster [DBG] pgmap v1184: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:13.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:13 smithi134 bash[18722]: audit 2024-02-10T08:41:12.526569+0000 mgr.y (mgr.14547) 721 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:14 smithi067 bash[17385]: cluster 2024-02-10T08:41:13.071877+0000 mon.a (mon.0) 3070 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T08:41:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:14 smithi067 bash[24488]: cluster 2024-02-10T08:41:13.071877+0000 mon.a (mon.0) 3070 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T08:41:14.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:14 smithi134 bash[18722]: cluster 2024-02-10T08:41:13.071877+0000 mon.a (mon.0) 3070 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-10T08:41:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:15 smithi067 bash[17385]: cluster 2024-02-10T08:41:14.313975+0000 mgr.y (mgr.14547) 722 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-02-10T08:41:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:15 smithi067 bash[24488]: cluster 2024-02-10T08:41:14.313975+0000 mgr.y (mgr.14547) 722 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-02-10T08:41:15.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:15 smithi134 bash[18722]: cluster 2024-02-10T08:41:14.313975+0000 mgr.y (mgr.14547) 722 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-02-10T08:41:17.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:16 smithi067 bash[17385]: audit 2024-02-10T08:41:15.797241+0000 mon.a (mon.0) 3071 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:17.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:16 smithi067 bash[24488]: audit 2024-02-10T08:41:15.797241+0000 mon.a (mon.0) 3071 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:17.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:16 smithi134 bash[18722]: audit 2024-02-10T08:41:15.797241+0000 mon.a (mon.0) 3071 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:17 smithi067 bash[17385]: cluster 2024-02-10T08:41:16.314787+0000 mgr.y (mgr.14547) 723 : cluster [DBG] pgmap v1186: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-02-10T08:41:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:41:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:17] "GET /metrics HTTP/1.1" 200 191664 "" "Prometheus/2.33.4" 2024-02-10T08:41:18.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:17 smithi067 bash[24488]: cluster 2024-02-10T08:41:16.314787+0000 mgr.y (mgr.14547) 723 : cluster [DBG] pgmap v1186: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-02-10T08:41:18.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:17 smithi134 bash[18722]: cluster 2024-02-10T08:41:16.314787+0000 mgr.y (mgr.14547) 723 : cluster [DBG] pgmap v1186: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-02-10T08:41:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:19 smithi067 bash[17385]: cluster 2024-02-10T08:41:18.316851+0000 mgr.y (mgr.14547) 724 : cluster [DBG] pgmap v1187: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:19 smithi067 bash[24488]: cluster 2024-02-10T08:41:18.316851+0000 mgr.y (mgr.14547) 724 : cluster [DBG] pgmap v1187: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:20.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:19 smithi134 bash[18722]: cluster 2024-02-10T08:41:18.316851+0000 mgr.y (mgr.14547) 724 : cluster [DBG] pgmap v1187: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:21 smithi067 bash[17385]: cluster 2024-02-10T08:41:20.317708+0000 mgr.y (mgr.14547) 725 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:21 smithi067 bash[17385]: audit 2024-02-10T08:41:21.530319+0000 mon.c (mon.2) 1011 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:21 smithi067 bash[17385]: audit 2024-02-10T08:41:21.530908+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:21 smithi067 bash[24488]: cluster 2024-02-10T08:41:20.317708+0000 mgr.y (mgr.14547) 725 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:21 smithi067 bash[24488]: audit 2024-02-10T08:41:21.530319+0000 mon.c (mon.2) 1011 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:22.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:21 smithi067 bash[24488]: audit 2024-02-10T08:41:21.530908+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:22.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:21 smithi134 bash[18722]: cluster 2024-02-10T08:41:20.317708+0000 mgr.y (mgr.14547) 725 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:22.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:21 smithi134 bash[18722]: audit 2024-02-10T08:41:21.530319+0000 mon.c (mon.2) 1011 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:22.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:21 smithi134 bash[18722]: audit 2024-02-10T08:41:21.530908+0000 mon.a (mon.0) 3072 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:22.832 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:41:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[17385]: audit 2024-02-10T08:41:21.820861+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[17385]: cluster 2024-02-10T08:41:21.820987+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-10T08:41:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[17385]: audit 2024-02-10T08:41:21.822552+0000 mon.c (mon.2) 1012 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[17385]: audit 2024-02-10T08:41:21.824879+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:23.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[17385]: audit 2024-02-10T08:41:22.822557+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]': finished 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[17385]: cluster 2024-02-10T08:41:22.822736+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[24488]: audit 2024-02-10T08:41:21.820861+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[24488]: cluster 2024-02-10T08:41:21.820987+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[24488]: audit 2024-02-10T08:41:21.822552+0000 mon.c (mon.2) 1012 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[24488]: audit 2024-02-10T08:41:21.824879+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[24488]: audit 2024-02-10T08:41:22.822557+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]': finished 2024-02-10T08:41:23.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:22 smithi067 bash[24488]: cluster 2024-02-10T08:41:22.822736+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-10T08:41:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[18722]: audit 2024-02-10T08:41:21.820861+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[18722]: cluster 2024-02-10T08:41:21.820987+0000 mon.a (mon.0) 3074 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-10T08:41:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[18722]: audit 2024-02-10T08:41:21.822552+0000 mon.c (mon.2) 1012 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[18722]: audit 2024-02-10T08:41:21.824879+0000 mon.a (mon.0) 3075 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[18722]: audit 2024-02-10T08:41:22.822557+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]': finished 2024-02-10T08:41:23.149 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:22 smithi134 bash[18722]: cluster 2024-02-10T08:41:22.822736+0000 mon.a (mon.0) 3077 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-10T08:41:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:41:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[24488]: cluster 2024-02-10T08:41:22.318564+0000 mgr.y (mgr.14547) 726 : cluster [DBG] pgmap v1190: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[24488]: audit 2024-02-10T08:41:22.533895+0000 mgr.y (mgr.14547) 727 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[24488]: audit 2024-02-10T08:41:22.865088+0000 mon.c (mon.2) 1013 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[24488]: audit 2024-02-10T08:41:22.865751+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[24488]: audit 2024-02-10T08:41:22.866705+0000 mon.c (mon.2) 1014 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[24488]: audit 2024-02-10T08:41:22.867253+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[17385]: cluster 2024-02-10T08:41:22.318564+0000 mgr.y (mgr.14547) 726 : cluster [DBG] pgmap v1190: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[17385]: audit 2024-02-10T08:41:22.533895+0000 mgr.y (mgr.14547) 727 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[17385]: audit 2024-02-10T08:41:22.865088+0000 mon.c (mon.2) 1013 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[17385]: audit 2024-02-10T08:41:22.865751+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[17385]: audit 2024-02-10T08:41:22.866705+0000 mon.c (mon.2) 1014 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:24.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:23 smithi067 bash[17385]: audit 2024-02-10T08:41:22.867253+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[18722]: cluster 2024-02-10T08:41:22.318564+0000 mgr.y (mgr.14547) 726 : cluster [DBG] pgmap v1190: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-10T08:41:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[18722]: audit 2024-02-10T08:41:22.533895+0000 mgr.y (mgr.14547) 727 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[18722]: audit 2024-02-10T08:41:22.865088+0000 mon.c (mon.2) 1013 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[18722]: audit 2024-02-10T08:41:22.865751+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[18722]: audit 2024-02-10T08:41:22.866705+0000 mon.c (mon.2) 1014 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:23 smithi134 bash[18722]: audit 2024-02-10T08:41:22.867253+0000 mon.a (mon.0) 3079 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-136"}]: dispatch 2024-02-10T08:41:25.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:24 smithi067 bash[17385]: cluster 2024-02-10T08:41:23.841297+0000 mon.a (mon.0) 3080 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-10T08:41:25.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:24 smithi067 bash[24488]: cluster 2024-02-10T08:41:23.841297+0000 mon.a (mon.0) 3080 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-10T08:41:25.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:24 smithi134 bash[18722]: cluster 2024-02-10T08:41:23.841297+0000 mon.a (mon.0) 3080 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-10T08:41:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[17385]: cluster 2024-02-10T08:41:24.319510+0000 mgr.y (mgr.14547) 728 : cluster [DBG] pgmap v1193: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T08:41:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[17385]: cluster 2024-02-10T08:41:24.850823+0000 mon.a (mon.0) 3081 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T08:41:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[17385]: cluster 2024-02-10T08:41:24.866956+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-10T08:41:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[17385]: audit 2024-02-10T08:41:24.872754+0000 mon.c (mon.2) 1015 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:26.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[17385]: audit 2024-02-10T08:41:24.873459+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[24488]: cluster 2024-02-10T08:41:24.319510+0000 mgr.y (mgr.14547) 728 : cluster [DBG] pgmap v1193: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T08:41:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[24488]: cluster 2024-02-10T08:41:24.850823+0000 mon.a (mon.0) 3081 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T08:41:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[24488]: cluster 2024-02-10T08:41:24.866956+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-10T08:41:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[24488]: audit 2024-02-10T08:41:24.872754+0000 mon.c (mon.2) 1015 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:26.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:25 smithi067 bash[24488]: audit 2024-02-10T08:41:24.873459+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:26.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:25 smithi134 bash[18722]: cluster 2024-02-10T08:41:24.319510+0000 mgr.y (mgr.14547) 728 : cluster [DBG] pgmap v1193: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-10T08:41:26.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:25 smithi134 bash[18722]: cluster 2024-02-10T08:41:24.850823+0000 mon.a (mon.0) 3081 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-10T08:41:26.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:25 smithi134 bash[18722]: cluster 2024-02-10T08:41:24.866956+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-10T08:41:26.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:25 smithi134 bash[18722]: audit 2024-02-10T08:41:24.872754+0000 mon.c (mon.2) 1015 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:26.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:25 smithi134 bash[18722]: audit 2024-02-10T08:41:24.873459+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:27.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:26 smithi134 bash[18722]: audit 2024-02-10T08:41:25.870279+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:27.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:26 smithi134 bash[18722]: cluster 2024-02-10T08:41:25.870461+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-10T08:41:27.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:26 smithi134 bash[18722]: audit 2024-02-10T08:41:25.901097+0000 mon.c (mon.2) 1016 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:27.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:26 smithi134 bash[18722]: audit 2024-02-10T08:41:25.901507+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[17385]: audit 2024-02-10T08:41:25.870279+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[17385]: cluster 2024-02-10T08:41:25.870461+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-10T08:41:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[17385]: audit 2024-02-10T08:41:25.901097+0000 mon.c (mon.2) 1016 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:27.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[17385]: audit 2024-02-10T08:41:25.901507+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[24488]: audit 2024-02-10T08:41:25.870279+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[24488]: cluster 2024-02-10T08:41:25.870461+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-10T08:41:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[24488]: audit 2024-02-10T08:41:25.901097+0000 mon.c (mon.2) 1016 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:27.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:26 smithi067 bash[24488]: audit 2024-02-10T08:41:25.901507+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-10T08:41:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:27] "GET /metrics HTTP/1.1" 200 191664 "" "Prometheus/2.33.4" 2024-02-10T08:41:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[17385]: cluster 2024-02-10T08:41:26.320383+0000 mgr.y (mgr.14547) 729 : cluster [DBG] pgmap v1196: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[17385]: audit 2024-02-10T08:41:26.878953+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[17385]: cluster 2024-02-10T08:41:26.879149+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[17385]: audit 2024-02-10T08:41:26.891564+0000 mon.c (mon.2) 1017 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[17385]: audit 2024-02-10T08:41:26.891952+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[24488]: cluster 2024-02-10T08:41:26.320383+0000 mgr.y (mgr.14547) 729 : cluster [DBG] pgmap v1196: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[24488]: audit 2024-02-10T08:41:26.878953+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[24488]: cluster 2024-02-10T08:41:26.879149+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[24488]: audit 2024-02-10T08:41:26.891564+0000 mon.c (mon.2) 1017 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:27 smithi067 bash[24488]: audit 2024-02-10T08:41:26.891952+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:28.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:27 smithi134 bash[18722]: cluster 2024-02-10T08:41:26.320383+0000 mgr.y (mgr.14547) 729 : cluster [DBG] pgmap v1196: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:28.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:27 smithi134 bash[18722]: audit 2024-02-10T08:41:26.878953+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-10T08:41:28.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:27 smithi134 bash[18722]: cluster 2024-02-10T08:41:26.879149+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-10T08:41:28.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:27 smithi134 bash[18722]: audit 2024-02-10T08:41:26.891564+0000 mon.c (mon.2) 1017 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:28.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:27 smithi134 bash[18722]: audit 2024-02-10T08:41:26.891952+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:29.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:28 smithi134 bash[18722]: audit 2024-02-10T08:41:27.891055+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]': finished 2024-02-10T08:41:29.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:28 smithi134 bash[18722]: cluster 2024-02-10T08:41:27.891167+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-10T08:41:29.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:28 smithi134 bash[18722]: audit 2024-02-10T08:41:27.937601+0000 mon.c (mon.2) 1018 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:28 smithi134 bash[18722]: audit 2024-02-10T08:41:27.938167+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:28 smithi134 bash[18722]: audit 2024-02-10T08:41:27.939514+0000 mon.c (mon.2) 1019 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:29.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:28 smithi134 bash[18722]: audit 2024-02-10T08:41:27.940065+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[17385]: audit 2024-02-10T08:41:27.891055+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]': finished 2024-02-10T08:41:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[17385]: cluster 2024-02-10T08:41:27.891167+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-10T08:41:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[17385]: audit 2024-02-10T08:41:27.937601+0000 mon.c (mon.2) 1018 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[17385]: audit 2024-02-10T08:41:27.938167+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[17385]: audit 2024-02-10T08:41:27.939514+0000 mon.c (mon.2) 1019 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:29.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[17385]: audit 2024-02-10T08:41:27.940065+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[24488]: audit 2024-02-10T08:41:27.891055+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]': finished 2024-02-10T08:41:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[24488]: cluster 2024-02-10T08:41:27.891167+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-10T08:41:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[24488]: audit 2024-02-10T08:41:27.937601+0000 mon.c (mon.2) 1018 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[24488]: audit 2024-02-10T08:41:27.938167+0000 mon.a (mon.0) 3092 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[24488]: audit 2024-02-10T08:41:27.939514+0000 mon.c (mon.2) 1019 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:29.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:28 smithi067 bash[24488]: audit 2024-02-10T08:41:27.940065+0000 mon.a (mon.0) 3093 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-138"}]: dispatch 2024-02-10T08:41:30.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:29 smithi134 bash[18722]: cluster 2024-02-10T08:41:28.321299+0000 mgr.y (mgr.14547) 730 : cluster [DBG] pgmap v1199: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:41:30.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:29 smithi134 bash[18722]: cluster 2024-02-10T08:41:28.895510+0000 mon.a (mon.0) 3094 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:30.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:29 smithi134 bash[18722]: cluster 2024-02-10T08:41:28.908469+0000 mon.a (mon.0) 3095 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-10T08:41:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:29 smithi067 bash[17385]: cluster 2024-02-10T08:41:28.321299+0000 mgr.y (mgr.14547) 730 : cluster [DBG] pgmap v1199: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:41:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:29 smithi067 bash[17385]: cluster 2024-02-10T08:41:28.895510+0000 mon.a (mon.0) 3094 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:30.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:29 smithi067 bash[17385]: cluster 2024-02-10T08:41:28.908469+0000 mon.a (mon.0) 3095 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-10T08:41:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:29 smithi067 bash[24488]: cluster 2024-02-10T08:41:28.321299+0000 mgr.y (mgr.14547) 730 : cluster [DBG] pgmap v1199: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-10T08:41:30.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:29 smithi067 bash[24488]: cluster 2024-02-10T08:41:28.895510+0000 mon.a (mon.0) 3094 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:30.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:29 smithi067 bash[24488]: cluster 2024-02-10T08:41:28.908469+0000 mon.a (mon.0) 3095 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-10T08:41:31.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:30 smithi134 bash[18722]: cluster 2024-02-10T08:41:29.913692+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-10T08:41:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:30 smithi134 bash[18722]: audit 2024-02-10T08:41:29.918454+0000 mon.c (mon.2) 1020 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:30 smithi134 bash[18722]: audit 2024-02-10T08:41:29.919681+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:30 smithi134 bash[18722]: cluster 2024-02-10T08:41:30.322223+0000 mgr.y (mgr.14547) 731 : cluster [DBG] pgmap v1202: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:41:31.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:30 smithi134 bash[18722]: audit 2024-02-10T08:41:30.795367+0000 mon.a (mon.0) 3098 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[17385]: cluster 2024-02-10T08:41:29.913692+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-10T08:41:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[17385]: audit 2024-02-10T08:41:29.918454+0000 mon.c (mon.2) 1020 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[17385]: audit 2024-02-10T08:41:29.919681+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[17385]: cluster 2024-02-10T08:41:30.322223+0000 mgr.y (mgr.14547) 731 : cluster [DBG] pgmap v1202: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:41:31.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[17385]: audit 2024-02-10T08:41:30.795367+0000 mon.a (mon.0) 3098 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[24488]: cluster 2024-02-10T08:41:29.913692+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-10T08:41:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[24488]: audit 2024-02-10T08:41:29.918454+0000 mon.c (mon.2) 1020 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[24488]: audit 2024-02-10T08:41:29.919681+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[24488]: cluster 2024-02-10T08:41:30.322223+0000 mgr.y (mgr.14547) 731 : cluster [DBG] pgmap v1202: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-10T08:41:31.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:30 smithi067 bash[24488]: audit 2024-02-10T08:41:30.795367+0000 mon.a (mon.0) 3098 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:31 smithi134 bash[18722]: audit 2024-02-10T08:41:30.924817+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:31 smithi134 bash[18722]: cluster 2024-02-10T08:41:30.925223+0000 mon.a (mon.0) 3100 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-10T08:41:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:31 smithi134 bash[18722]: audit 2024-02-10T08:41:30.972694+0000 mon.c (mon.2) 1021 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:31 smithi134 bash[18722]: audit 2024-02-10T08:41:30.973242+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:31 smithi134 bash[18722]: audit 2024-02-10T08:41:30.974162+0000 mon.c (mon.2) 1022 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-140"}]: dispatch 2024-02-10T08:41:32.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:31 smithi134 bash[18722]: audit 2024-02-10T08:41:30.974689+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-140"}]: dispatch 2024-02-10T08:41:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[17385]: audit 2024-02-10T08:41:30.924817+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[17385]: cluster 2024-02-10T08:41:30.925223+0000 mon.a (mon.0) 3100 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-10T08:41:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[17385]: audit 2024-02-10T08:41:30.972694+0000 mon.c (mon.2) 1021 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[17385]: audit 2024-02-10T08:41:30.973242+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:32.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[17385]: audit 2024-02-10T08:41:30.974162+0000 mon.c (mon.2) 1022 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-140"}]: dispatch 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[17385]: audit 2024-02-10T08:41:30.974689+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-140"}]: dispatch 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[24488]: audit 2024-02-10T08:41:30.924817+0000 mon.a (mon.0) 3099 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[24488]: cluster 2024-02-10T08:41:30.925223+0000 mon.a (mon.0) 3100 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[24488]: audit 2024-02-10T08:41:30.972694+0000 mon.c (mon.2) 1021 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[24488]: audit 2024-02-10T08:41:30.973242+0000 mon.a (mon.0) 3101 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[24488]: audit 2024-02-10T08:41:30.974162+0000 mon.c (mon.2) 1022 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-140"}]: dispatch 2024-02-10T08:41:32.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:31 smithi067 bash[24488]: audit 2024-02-10T08:41:30.974689+0000 mon.a (mon.0) 3102 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-140"}]: dispatch 2024-02-10T08:41:32.945 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:41:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:41:33.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:32 smithi134 bash[18722]: cluster 2024-02-10T08:41:31.928410+0000 mon.a (mon.0) 3103 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-10T08:41:33.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:32 smithi134 bash[18722]: cluster 2024-02-10T08:41:32.323027+0000 mgr.y (mgr.14547) 732 : cluster [DBG] pgmap v1205: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:41:33.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:32 smithi134 bash[18722]: audit 2024-02-10T08:41:32.543055+0000 mgr.y (mgr.14547) 733 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:33.277 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:41:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:41:33.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:32 smithi067 bash[17385]: cluster 2024-02-10T08:41:31.928410+0000 mon.a (mon.0) 3103 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-10T08:41:33.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:32 smithi067 bash[17385]: cluster 2024-02-10T08:41:32.323027+0000 mgr.y (mgr.14547) 732 : cluster [DBG] pgmap v1205: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:41:33.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:32 smithi067 bash[17385]: audit 2024-02-10T08:41:32.543055+0000 mgr.y (mgr.14547) 733 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:32 smithi067 bash[24488]: cluster 2024-02-10T08:41:31.928410+0000 mon.a (mon.0) 3103 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-10T08:41:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:32 smithi067 bash[24488]: cluster 2024-02-10T08:41:32.323027+0000 mgr.y (mgr.14547) 732 : cluster [DBG] pgmap v1205: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-10T08:41:33.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:32 smithi067 bash[24488]: audit 2024-02-10T08:41:32.543055+0000 mgr.y (mgr.14547) 733 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:33 smithi134 bash[18722]: cluster 2024-02-10T08:41:32.947636+0000 mon.a (mon.0) 3104 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-10T08:41:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:33 smithi134 bash[18722]: audit 2024-02-10T08:41:32.951430+0000 mon.c (mon.2) 1023 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:33 smithi134 bash[18722]: audit 2024-02-10T08:41:32.952200+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:33 smithi067 bash[17385]: cluster 2024-02-10T08:41:32.947636+0000 mon.a (mon.0) 3104 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-10T08:41:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:33 smithi067 bash[17385]: audit 2024-02-10T08:41:32.951430+0000 mon.c (mon.2) 1023 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:34.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:33 smithi067 bash[17385]: audit 2024-02-10T08:41:32.952200+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:33 smithi067 bash[24488]: cluster 2024-02-10T08:41:32.947636+0000 mon.a (mon.0) 3104 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-10T08:41:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:33 smithi067 bash[24488]: audit 2024-02-10T08:41:32.951430+0000 mon.c (mon.2) 1023 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:34.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:33 smithi067 bash[24488]: audit 2024-02-10T08:41:32.952200+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:35.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: audit 2024-02-10T08:41:33.958896+0000 mon.a (mon.0) 3106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:35.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: cluster 2024-02-10T08:41:33.959111+0000 mon.a (mon.0) 3107 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-10T08:41:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: audit 2024-02-10T08:41:34.027544+0000 mon.c (mon.2) 1024 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: audit 2024-02-10T08:41:34.028225+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: audit 2024-02-10T08:41:34.029322+0000 mon.c (mon.2) 1025 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-142"}]: dispatch 2024-02-10T08:41:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: audit 2024-02-10T08:41:34.029843+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-142"}]: dispatch 2024-02-10T08:41:35.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:34 smithi134 bash[18722]: cluster 2024-02-10T08:41:34.323839+0000 mgr.y (mgr.14547) 734 : cluster [DBG] pgmap v1208: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: audit 2024-02-10T08:41:33.958896+0000 mon.a (mon.0) 3106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: cluster 2024-02-10T08:41:33.959111+0000 mon.a (mon.0) 3107 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: audit 2024-02-10T08:41:34.027544+0000 mon.c (mon.2) 1024 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: audit 2024-02-10T08:41:34.028225+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: audit 2024-02-10T08:41:34.029322+0000 mon.c (mon.2) 1025 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-142"}]: dispatch 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: audit 2024-02-10T08:41:34.029843+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-142"}]: dispatch 2024-02-10T08:41:35.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[17385]: cluster 2024-02-10T08:41:34.323839+0000 mgr.y (mgr.14547) 734 : cluster [DBG] pgmap v1208: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: audit 2024-02-10T08:41:33.958896+0000 mon.a (mon.0) 3106 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: cluster 2024-02-10T08:41:33.959111+0000 mon.a (mon.0) 3107 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: audit 2024-02-10T08:41:34.027544+0000 mon.c (mon.2) 1024 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: audit 2024-02-10T08:41:34.028225+0000 mon.a (mon.0) 3108 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: audit 2024-02-10T08:41:34.029322+0000 mon.c (mon.2) 1025 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-142"}]: dispatch 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: audit 2024-02-10T08:41:34.029843+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-142"}]: dispatch 2024-02-10T08:41:35.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:34 smithi067 bash[24488]: cluster 2024-02-10T08:41:34.323839+0000 mgr.y (mgr.14547) 734 : cluster [DBG] pgmap v1208: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-10T08:41:36.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:35 smithi134 bash[18722]: cluster 2024-02-10T08:41:34.968617+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-10T08:41:36.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:35 smithi067 bash[17385]: cluster 2024-02-10T08:41:34.968617+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-10T08:41:36.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:35 smithi067 bash[24488]: cluster 2024-02-10T08:41:34.968617+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-10T08:41:37.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:36 smithi134 bash[18722]: cluster 2024-02-10T08:41:35.978878+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-10T08:41:37.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:36 smithi134 bash[18722]: audit 2024-02-10T08:41:35.985385+0000 mon.c (mon.2) 1026 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:36 smithi134 bash[18722]: audit 2024-02-10T08:41:35.988005+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:37.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:36 smithi134 bash[18722]: cluster 2024-02-10T08:41:36.324663+0000 mgr.y (mgr.14547) 735 : cluster [DBG] pgmap v1211: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[24488]: cluster 2024-02-10T08:41:35.978878+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-10T08:41:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[24488]: audit 2024-02-10T08:41:35.985385+0000 mon.c (mon.2) 1026 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[24488]: audit 2024-02-10T08:41:35.988005+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:37.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[24488]: cluster 2024-02-10T08:41:36.324663+0000 mgr.y (mgr.14547) 735 : cluster [DBG] pgmap v1211: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:37.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[17385]: cluster 2024-02-10T08:41:35.978878+0000 mon.a (mon.0) 3111 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-10T08:41:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[17385]: audit 2024-02-10T08:41:35.985385+0000 mon.c (mon.2) 1026 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[17385]: audit 2024-02-10T08:41:35.988005+0000 mon.a (mon.0) 3112 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-10T08:41:37.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:36 smithi067 bash[17385]: cluster 2024-02-10T08:41:36.324663+0000 mgr.y (mgr.14547) 735 : cluster [DBG] pgmap v1211: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:38.157 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:37] "GET /metrics HTTP/1.1" 200 191767 "" "Prometheus/2.33.4" 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:36.991424+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: cluster 2024-02-10T08:41:36.991626+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:36.995000+0000 mon.c (mon.2) 1027 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-101","var": "dedup_tier","val": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:36.998000+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-101","var": "dedup_tier","val": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:37.028172+0000 mon.c (mon.2) 1028 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:37.028749+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:38.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:37.029732+0000 mon.c (mon.2) 1029 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: audit 2024-02-10T08:41:37.030171+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[17385]: cluster 2024-02-10T08:41:37.410396+0000 mon.a (mon.0) 3118 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:36.991424+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: cluster 2024-02-10T08:41:36.991626+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:36.995000+0000 mon.c (mon.2) 1027 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-101","var": "dedup_tier","val": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:36.998000+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-101","var": "dedup_tier","val": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:37.028172+0000 mon.c (mon.2) 1028 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:37.028749+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:37.029732+0000 mon.c (mon.2) 1029 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: audit 2024-02-10T08:41:37.030171+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.159 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:37 smithi067 bash[24488]: cluster 2024-02-10T08:41:37.410396+0000 mon.a (mon.0) 3118 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:36.991424+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi067-54339-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-10T08:41:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: cluster 2024-02-10T08:41:36.991626+0000 mon.a (mon.0) 3114 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-10T08:41:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:36.995000+0000 mon.c (mon.2) 1027 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-101","var": "dedup_tier","val": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:36.998000+0000 mon.a (mon.0) 3115 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi067-54339-101","var": "dedup_tier","val": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:37.028172+0000 mon.c (mon.2) 1028 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:37.028749+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:37.029732+0000 mon.c (mon.2) 1029 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: audit 2024-02-10T08:41:37.030171+0000 mon.a (mon.0) 3117 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi067-54339-101", "tierpool": "test-rados-api-smithi067-54339-144"}]: dispatch 2024-02-10T08:41:38.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:37 smithi134 bash[18722]: cluster 2024-02-10T08:41:37.410396+0000 mon.a (mon.0) 3118 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:39.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:39 smithi134 bash[18722]: cluster 2024-02-10T08:41:37.998003+0000 mon.a (mon.0) 3119 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-10T08:41:39.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:39 smithi134 bash[18722]: cluster 2024-02-10T08:41:38.325593+0000 mgr.y (mgr.14547) 736 : cluster [DBG] pgmap v1214: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-10T08:41:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:39 smithi067 bash[17385]: cluster 2024-02-10T08:41:37.998003+0000 mon.a (mon.0) 3119 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-10T08:41:39.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:39 smithi067 bash[17385]: cluster 2024-02-10T08:41:38.325593+0000 mgr.y (mgr.14547) 736 : cluster [DBG] pgmap v1214: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-10T08:41:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:39 smithi067 bash[24488]: cluster 2024-02-10T08:41:37.998003+0000 mon.a (mon.0) 3119 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-10T08:41:39.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:39 smithi067 bash[24488]: cluster 2024-02-10T08:41:38.325593+0000 mgr.y (mgr.14547) 736 : cluster [DBG] pgmap v1214: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-10T08:41:40.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:40 smithi134 bash[18722]: cluster 2024-02-10T08:41:39.006908+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-10T08:41:40.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:40 smithi134 bash[18722]: audit 2024-02-10T08:41:39.010393+0000 mon.c (mon.2) 1030 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:40.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:40 smithi134 bash[18722]: audit 2024-02-10T08:41:39.010988+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:40 smithi067 bash[17385]: cluster 2024-02-10T08:41:39.006908+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-10T08:41:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:40 smithi067 bash[17385]: audit 2024-02-10T08:41:39.010393+0000 mon.c (mon.2) 1030 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:40.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:40 smithi067 bash[17385]: audit 2024-02-10T08:41:39.010988+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:40.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:40 smithi067 bash[24488]: cluster 2024-02-10T08:41:39.006908+0000 mon.a (mon.0) 3120 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-10T08:41:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:40 smithi067 bash[24488]: audit 2024-02-10T08:41:39.010393+0000 mon.c (mon.2) 1030 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:40.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:40 smithi067 bash[24488]: audit 2024-02-10T08:41:39.010988+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.021 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9096 ms) 2024-02-10T08:41:41.021 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-10T08:41:41.021 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: first is 1707554416 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,1707554421,1707554422,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,1707554421,1707554422,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,1707554421,1707554422,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,1707554421,1707554422,1707554424,1707554425,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,1707554421,1707554422,1707554424,1707554425,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554416,1707554418,1707554419,1707554421,1707554422,1707554424,1707554425,0 2024-02-10T08:41:41.022 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: got ls 1707554419,1707554421,1707554422,1707554424,1707554425,1707554427,1707554428,0 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: first now 1707554419, trimmed 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21144 ms) 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: foo0 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14197 ms) 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17822 ms) 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22442 ms) 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5064 ms) 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3020 ms) 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-10T08:41:41.023 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3042 ms) 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3030 ms) 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (231656 ms total) 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [==========] 72 tests from 4 test suites ran. (804178 ms total) 2024-02-10T08:41:41.024 INFO:tasks.workunit.client.0.smithi067.stdout: api_tier_pp: [ PASSED ] 72 tests. 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54506 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54506 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54280 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54280 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54264 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54264 2024-02-10T08:41:41.026 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54296 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54296 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54529 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54529 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54679 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54679 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54457 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54457 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54240 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54240 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.027 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54749 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54749 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54620 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54620 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54497 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54497 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54715 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54715 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54272 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54272 2024-02-10T08:41:41.028 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54631 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54631 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54256 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54256 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54574 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54574 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54309 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54309 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54596 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54596 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ for t in "${!pids[@]}" 2024-02-10T08:41:41.029 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=54479 2024-02-10T08:41:41.030 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 54479 2024-02-10T08:41:41.030 INFO:tasks.workunit.client.0.smithi067.stderr:+ exit 0 2024-02-10T08:41:41.030 INFO:tasks.workunit.client.0.smithi067.stderr:+ cleanup 2024-02-10T08:41:41.030 INFO:tasks.workunit.client.0.smithi067.stderr:+ pkill -P 54227 2024-02-10T08:41:41.039 INFO:tasks.workunit.client.0.smithi067.stderr:+ true 2024-02-10T08:41:41.039 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-10T08:41:41.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-10T08:41:41.057 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-10T08:41:41.058 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-10T08:41:41.058 DEBUG:teuthology.orchestra.run.smithi067:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57bd6abdec7bb457ae7999d9c96682e9ac678e27 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-10T08:41:41.110 INFO:tasks.workunit.client.0.smithi067.stderr:+ uuidgen 2024-02-10T08:41:41.111 INFO:tasks.workunit.client.0.smithi067.stderr:+ p=34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:41:41.111 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool create 34c11e82-d78d-4916-9c14-acce32e59950 12 2024-02-10T08:41:41.218 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 -- 172.21.15.67:0/3239460689 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 msgr2=0x7f65a8106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:41.218 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 --2- 172.21.15.67:0/3239460689 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8106350 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 rx=0x7f65a0009be0 tx=0x7f65a0004d10).stop 2024-02-10T08:41:41.218 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 -- 172.21.15.67:0/3239460689 shutdown_connections 2024-02-10T08:41:41.219 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 --2- 172.21.15.67:0/3239460689 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8106350 unknown :-1 s=CLOSED pgs=3036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:41.219 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 --2- 172.21.15.67:0/3239460689 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a806b560 0x7f65a8103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:41.219 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 --2- 172.21.15.67:0/3239460689 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:41.219 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 -- 172.21.15.67:0/3239460689 >> 172.21.15.67:0/3239460689 conn(0x7f65a80f3360 msgr2=0x7f65a80f5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:41:41.219 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 -- 172.21.15.67:0/3239460689 shutdown_connections 2024-02-10T08:41:41.219 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.210+0000 7f65afe0c700 1 -- 172.21.15.67:0/3239460689 wait complete. 2024-02-10T08:41:41.221 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 Processor -- start 2024-02-10T08:41:41.221 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 -- start start 2024-02-10T08:41:41.221 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a810f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:41.221 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a806b560 0x7f65a810fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:41.221 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a810a0c0 con 0x7f65a806ac90 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a810a230 con 0x7f65a806b560 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a810a3f0 con 0x7f65a8103f60 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a810f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a810f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39298/0 (socket says 172.21.15.67:39298) 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 -- 172.21.15.67:0/4278525572 learned_addr learned my addr 172.21.15.67:0/4278525572 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65ae3a9700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65ad3a7700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a806b560 0x7f65a810fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 -- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 msgr2=0x7f65a8109bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8109bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 -- 172.21.15.67:0/4278525572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a806b560 msgr2=0x7f65a810fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a806b560 0x7f65a810fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 -- 172.21.15.67:0/4278525572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00097c0 con 0x7f65a806ac90 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65ae3a9700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:41:41.222 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65adba8700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a810f800 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 rx=0x7f65a400e970 tx=0x7f65a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4004660 con 0x7f65a806ac90 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f65a400f040 con 0x7f65a806ac90 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a810a6d0 con 0x7f65a806ac90 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a40146f0 con 0x7f65a806ac90 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.214+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a810aa50 con 0x7f65a806ac90 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.218+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a8060470 con 0x7f65a806ac90 2024-02-10T08:41:41.223 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.218+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f65a4007b40 con 0x7f65a806ac90 2024-02-10T08:41:41.224 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.218+0000 7f659effd700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6594061340 0x7f6594063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:41.224 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.218+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(715..715 src has 1..715) v4 ==== 8395+0+0 (secure 0 0 0) 0x7f65a407ce00 con 0x7f65a806ac90 2024-02-10T08:41:41.224 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.218+0000 7f65ad3a7700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6594061340 0x7f6594063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:41.224 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.218+0000 7f65ad3a7700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6594061340 0x7f6594063800 secure :-1 s=READY pgs=3559 cs=0 l=1 rev1=1 rx=0x7f65a810b160 tx=0x7f659800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:41:41.232 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.226+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f65a404c800 con 0x7f65a806ac90 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: audit 2024-02-10T08:41:40.011128+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: cluster 2024-02-10T08:41:40.011381+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: audit 2024-02-10T08:41:40.012854+0000 mon.c (mon.2) 1031 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: audit 2024-02-10T08:41:40.014752+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: cluster 2024-02-10T08:41:40.326442+0000 mgr.y (mgr.14547) 737 : cluster [DBG] pgmap v1217: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: audit 2024-02-10T08:41:41.013830+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:41.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:41 smithi134 bash[18722]: cluster 2024-02-10T08:41:41.013940+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: audit 2024-02-10T08:41:40.011128+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: cluster 2024-02-10T08:41:40.011381+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: audit 2024-02-10T08:41:40.012854+0000 mon.c (mon.2) 1031 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: audit 2024-02-10T08:41:40.014752+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: cluster 2024-02-10T08:41:40.326442+0000 mgr.y (mgr.14547) 737 : cluster [DBG] pgmap v1217: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: audit 2024-02-10T08:41:41.013830+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:41.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[24488]: cluster 2024-02-10T08:41:41.013940+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: audit 2024-02-10T08:41:40.011128+0000 mon.a (mon.0) 3122 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: cluster 2024-02-10T08:41:40.011381+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: audit 2024-02-10T08:41:40.012854+0000 mon.c (mon.2) 1031 : audit [INF] from='client.? 172.21.15.67:0/3485854985' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: audit 2024-02-10T08:41:40.014752+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]: dispatch 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: cluster 2024-02-10T08:41:40.326442+0000 mgr.y (mgr.14547) 737 : cluster [DBG] pgmap v1217: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: audit 2024-02-10T08:41:41.013830+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi067-54339-101"}]': finished 2024-02-10T08:41:41.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:41 smithi067 bash[17385]: cluster 2024-02-10T08:41:41.013940+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-10T08:41:41.388 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:41.382+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12} v 0) v1 -- 0x7f65a805fc90 con 0x7f65a806ac90 2024-02-10T08:41:42.042 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.034+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]=0 pool '34c11e82-d78d-4916-9c14-acce32e59950' created v716) v1 ==== 176+0+0 (secure 0 0 0) 0x7f65a4019070 con 0x7f65a806ac90 2024-02-10T08:41:42.143 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12} v 0) v1 -- 0x7f65a8108c60 con 0x7f65a806ac90 2024-02-10T08:41:42.144 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f659effd700 1 -- 172.21.15.67:0/4278525572 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]=0 pool '34c11e82-d78d-4916-9c14-acce32e59950' already exists v716) v1 ==== 183+0+0 (secure 0 0 0) 0x7f65a404fe20 con 0x7f65a806ac90 2024-02-10T08:41:42.146 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6594061340 msgr2=0x7f6594063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6594061340 0x7f6594063800 secure :-1 s=READY pgs=3559 cs=0 l=1 rev1=1 rx=0x7f65a810b160 tx=0x7f659800b040).stop 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 msgr2=0x7f65a810f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a810f800 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 rx=0x7f65a400e970 tx=0x7f65a400b040).stop 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 shutdown_connections 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6594061340 0x7f6594063800 unknown :-1 s=CLOSED pgs=3559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f65a8103f60 0x7f65a8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65a806b560 0x7f65a810fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 --2- 172.21.15.67:0/4278525572 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f65a806ac90 0x7f65a810f800 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.138+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 >> 172.21.15.67:0/4278525572 conn(0x7f65a80f3360 msgr2=0x7f65a80f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.142+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 shutdown_connections 2024-02-10T08:41:42.147 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.142+0000 7f65afe0c700 1 -- 172.21.15.67:0/4278525572 wait complete. 2024-02-10T08:41:42.148 INFO:tasks.workunit.client.0.smithi067.stderr:pool '34c11e82-d78d-4916-9c14-acce32e59950' already exists 2024-02-10T08:41:42.156 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_objects 10 2024-02-10T08:41:42.261 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 -- 172.21.15.67:0/4222905004 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000f6220 msgr2=0x7f16000f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 --2- 172.21.15.67:0/4222905004 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000f6220 0x7f16000f6680 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 rx=0x7f15f8004870 tx=0x7f15f8009cb0).stop 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 -- 172.21.15.67:0/4222905004 shutdown_connections 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 --2- 172.21.15.67:0/4222905004 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000f6220 0x7f16000f6680 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 --2- 172.21.15.67:0/4222905004 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000fdfb0 0x7f16000f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 --2- 172.21.15.67:0/4222905004 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16000fd6e0 0x7f16000fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 -- 172.21.15.67:0/4222905004 >> 172.21.15.67:0/4222905004 conn(0x7f160006fc20 msgr2=0x7f16000f3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 -- 172.21.15.67:0/4222905004 shutdown_connections 2024-02-10T08:41:42.262 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 -- 172.21.15.67:0/4222905004 wait complete. 2024-02-10T08:41:42.264 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 Processor -- start 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.254+0000 7f1605aff700 1 -- start start 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000f6220 0x7f160010d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16000fd6e0 0x7f160010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000fdfb0 0x7f1600107ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600107fb0 con 0x7f16000f6220 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600108120 con 0x7f16000fd6e0 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16001082e0 con 0x7f16000fdfb0 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fffff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000fdfb0 0x7f1600107ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000f6220 0x7f160010d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fffff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000fdfb0 0x7f1600107ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:43428/0 (socket says 172.21.15.67:43428) 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fffff700 1 -- 172.21.15.67:0/170025336 learned_addr learned my addr 172.21.15.67:0/170025336 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:41:42.265 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15feffd700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16000fd6e0 0x7f160010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 -- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000fdfb0 msgr2=0x7f1600107ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000fdfb0 0x7f1600107ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 -- 172.21.15.67:0/170025336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16000fd6e0 msgr2=0x7f160010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16000fd6e0 0x7f160010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 -- 172.21.15.67:0/170025336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8005040 con 0x7f16000f6220 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15ff7fe700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000f6220 0x7f160010d900 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 rx=0x7f1600060670 tx=0x7f15f000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0009890 con 0x7f16000f6220 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f8005040 con 0x7f16000f6220 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1600114720 con 0x7f16000f6220 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f15f000f040 con 0x7f16000f6220 2024-02-10T08:41:42.266 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f000c5b0 con 0x7f16000f6220 2024-02-10T08:41:42.267 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e0004f50 con 0x7f16000f6220 2024-02-10T08:41:42.267 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.258+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f15f00094f0 con 0x7f16000f6220 2024-02-10T08:41:42.267 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.262+0000 7f15fcff9700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f15ec061340 0x7f15ec063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:42.267 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.262+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(716..716 src has 1..716) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f15f007d140 con 0x7f16000f6220 2024-02-10T08:41:42.268 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.262+0000 7f15feffd700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f15ec061340 0x7f15ec063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:42.268 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.262+0000 7f15feffd700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f15ec061340 0x7f15ec063800 secure :-1 s=READY pgs=3560 cs=0 l=1 rev1=1 rx=0x7f1600108dc0 tx=0x7f15e800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:41:42.273 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.266+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f15f004af60 con 0x7f16000f6220 2024-02-10T08:41:42.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:42 smithi067 bash[17385]: audit 2024-02-10T08:41:41.389294+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]: dispatch 2024-02-10T08:41:42.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:42 smithi067 bash[24488]: audit 2024-02-10T08:41:41.389294+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]: dispatch 2024-02-10T08:41:42.431 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:42.426+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f15e0005620 con 0x7f16000f6220 2024-02-10T08:41:42.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:42 smithi134 bash[18722]: audit 2024-02-10T08:41:41.389294+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]: dispatch 2024-02-10T08:41:43.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:41:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:41:43.052 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:43.046+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v717) v1 ==== 223+0+0 (secure 0 0 0) 0x7f15f0019070 con 0x7f16000f6220 2024-02-10T08:41:43.155 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:43.150+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f15e0005d50 con 0x7f16000f6220 2024-02-10T08:41:43.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: audit 2024-02-10T08:41:42.038731+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]': finished 2024-02-10T08:41:43.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: cluster 2024-02-10T08:41:42.038975+0000 mon.a (mon.0) 3129 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-10T08:41:43.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: audit 2024-02-10T08:41:42.143942+0000 mon.a (mon.0) 3130 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]: dispatch 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: cluster 2024-02-10T08:41:42.327200+0000 mgr.y (mgr.14547) 738 : cluster [DBG] pgmap v1220: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: cluster 2024-02-10T08:41:42.412124+0000 mon.a (mon.0) 3131 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: audit 2024-02-10T08:41:42.431686+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[17385]: audit 2024-02-10T08:41:42.551037+0000 mgr.y (mgr.14547) 739 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: audit 2024-02-10T08:41:42.038731+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]': finished 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: cluster 2024-02-10T08:41:42.038975+0000 mon.a (mon.0) 3129 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: audit 2024-02-10T08:41:42.143942+0000 mon.a (mon.0) 3130 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]: dispatch 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: cluster 2024-02-10T08:41:42.327200+0000 mgr.y (mgr.14547) 738 : cluster [DBG] pgmap v1220: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: cluster 2024-02-10T08:41:42.412124+0000 mon.a (mon.0) 3131 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: audit 2024-02-10T08:41:42.431686+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:41:43.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:43 smithi067 bash[24488]: audit 2024-02-10T08:41:42.551037+0000 mgr.y (mgr.14547) 739 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:41:43.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: audit 2024-02-10T08:41:42.038731+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]': finished 2024-02-10T08:41:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: cluster 2024-02-10T08:41:42.038975+0000 mon.a (mon.0) 3129 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-10T08:41:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: audit 2024-02-10T08:41:42.143942+0000 mon.a (mon.0) 3130 : audit [INF] from='client.? 172.21.15.67:0/4278525572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pg_num": 12}]: dispatch 2024-02-10T08:41:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: cluster 2024-02-10T08:41:42.327200+0000 mgr.y (mgr.14547) 738 : cluster [DBG] pgmap v1220: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: cluster 2024-02-10T08:41:42.412124+0000 mon.a (mon.0) 3131 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-10T08:41:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: audit 2024-02-10T08:41:42.431686+0000 mon.a (mon.0) 3132 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:41:43.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:43 smithi134 bash[18722]: audit 2024-02-10T08:41:42.551037+0000 mgr.y (mgr.14547) 739 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:44.056 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.054+0000 7f15fcff9700 1 -- 172.21.15.67:0/170025336 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v718) v1 ==== 223+0+0 (secure 0 0 0) 0x7f15f004c920 con 0x7f16000f6220 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f15ec061340 msgr2=0x7f15ec063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f15ec061340 0x7f15ec063800 secure :-1 s=READY pgs=3560 cs=0 l=1 rev1=1 rx=0x7f1600108dc0 tx=0x7f15e800b040).stop 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000f6220 msgr2=0x7f160010d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000f6220 0x7f160010d900 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 rx=0x7f1600060670 tx=0x7f15f000b040).stop 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 shutdown_connections 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f15ec061340 0x7f15ec063800 unknown :-1 s=CLOSED pgs=3560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f16000fdfb0 0x7f1600107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16000fd6e0 0x7f160010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.062 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 --2- 172.21.15.67:0/170025336 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f16000f6220 0x7f160010d900 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.062 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 >> 172.21.15.67:0/170025336 conn(0x7f160006fc20 msgr2=0x7f16000fb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:41:44.062 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 shutdown_connections 2024-02-10T08:41:44.062 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.058+0000 7f1605aff700 1 -- 172.21.15.67:0/170025336 wait complete. 2024-02-10T08:41:44.062 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_objects = 10 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:41:44.073 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool application enable 34c11e82-d78d-4916-9c14-acce32e59950 rados 2024-02-10T08:41:44.178 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.174+0000 7f96df9dc700 1 -- 172.21.15.67:0/1147844095 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d8101cb0 msgr2=0x7f96d81040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.174+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1147844095 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d8101cb0 0x7f96d81040a0 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 rx=0x7f96d0009c10 tx=0x7f96d0004d10).stop 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.174+0000 7f96df9dc700 1 -- 172.21.15.67:0/1147844095 shutdown_connections 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.174+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1147844095 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d8101cb0 0x7f96d81040a0 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1147844095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d80f90b0 0x7f96d81017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1147844095 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d80f8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- 172.21.15.67:0/1147844095 >> 172.21.15.67:0/1147844095 conn(0x7f96d806e610 msgr2=0x7f96d80f48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- 172.21.15.67:0/1147844095 shutdown_connections 2024-02-10T08:41:44.179 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- 172.21.15.67:0/1147844095 wait complete. 2024-02-10T08:41:44.180 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 Processor -- start 2024-02-10T08:41:44.180 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- start start 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d810d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d80f90b0 0x7f96d810dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8101cb0 0x7f96d81079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8107e90 con 0x7f96d80f8750 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8108000 con 0x7f96d8101cb0 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d81081c0 con 0x7f96d80f90b0 2024-02-10T08:41:44.181 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d810d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d810d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:39572/0 (socket says 172.21.15.67:39572) 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 -- 172.21.15.67:0/1878446500 learned_addr learned my addr 172.21.15.67:0/1878446500 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dcf77700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d80f90b0 0x7f96d810dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96ddf79700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8101cb0 0x7f96d81079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 -- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d80f90b0 msgr2=0x7f96d810dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d80f90b0 0x7f96d810dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 -- 172.21.15.67:0/1878446500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8101cb0 msgr2=0x7f96d81079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8101cb0 0x7f96d81079a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 -- 172.21.15.67:0/1878446500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00097c0 con 0x7f96d80f8750 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96ddf79700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8101cb0 0x7f96d81079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dcf77700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d80f90b0 0x7f96d810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:41:44.182 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96dd778700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d810d7e0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f96d400e970 tx=0x7f96d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:41:44.183 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400ee60 con 0x7f96d80f8750 2024-02-10T08:41:44.183 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d81084a0 con 0x7f96d80f8750 2024-02-10T08:41:44.183 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f96d400f040 con 0x7f96d80f8750 2024-02-10T08:41:44.183 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d400c5b0 con 0x7f96d80f8750 2024-02-10T08:41:44.183 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.178+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d8108820 con 0x7f96d80f8750 2024-02-10T08:41:44.184 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.182+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d805fc90 con 0x7f96d80f8750 2024-02-10T08:41:44.184 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.182+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f96d4007b40 con 0x7f96d80f8750 2024-02-10T08:41:44.185 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.182+0000 7f96ce7fc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f96c4061340 0x7f96c4063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:41:44.185 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.182+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(718..718 src has 1..718) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f96d407d160 con 0x7f96d80f8750 2024-02-10T08:41:44.185 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.182+0000 7f96dcf77700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f96c4061340 0x7f96c4063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:41:44.186 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.182+0000 7f96dcf77700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f96c4061340 0x7f96c4063800 secure :-1 s=READY pgs=3561 cs=0 l=1 rev1=1 rx=0x7f96d8108ca0 tx=0x7f96c800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:41:44.193 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.190+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f96d404af60 con 0x7f96d80f8750 2024-02-10T08:41:44.349 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:44.346+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"} v 0) v1 -- 0x7f96d8108e80 con 0x7f96d80f8750 2024-02-10T08:41:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:44 smithi067 bash[17385]: audit 2024-02-10T08:41:43.050888+0000 mon.a (mon.0) 3133 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:41:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:44 smithi067 bash[17385]: cluster 2024-02-10T08:41:43.050995+0000 mon.a (mon.0) 3134 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-10T08:41:44.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:44 smithi067 bash[17385]: audit 2024-02-10T08:41:43.155446+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:41:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:44 smithi067 bash[24488]: audit 2024-02-10T08:41:43.050888+0000 mon.a (mon.0) 3133 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:41:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:44 smithi067 bash[24488]: cluster 2024-02-10T08:41:43.050995+0000 mon.a (mon.0) 3134 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-10T08:41:44.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:44 smithi067 bash[24488]: audit 2024-02-10T08:41:43.155446+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:41:44.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:44 smithi134 bash[18722]: audit 2024-02-10T08:41:43.050888+0000 mon.a (mon.0) 3133 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:41:44.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:44 smithi134 bash[18722]: cluster 2024-02-10T08:41:43.050995+0000 mon.a (mon.0) 3134 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-10T08:41:44.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:44 smithi134 bash[18722]: audit 2024-02-10T08:41:43.155446+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:41:45.075 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:45.074+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]=0 enabled application 'rados' on pool '34c11e82-d78d-4916-9c14-acce32e59950' v719) v1 ==== 213+0+0 (secure 0 0 0) 0x7f96d4019070 con 0x7f96d80f8750 2024-02-10T08:41:45.177 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:45.174+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"} v 0) v1 -- 0x7f96d8114c80 con 0x7f96d80f8750 2024-02-10T08:41:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[24488]: audit 2024-02-10T08:41:44.055643+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:41:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[24488]: cluster 2024-02-10T08:41:44.055856+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-10T08:41:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[24488]: cluster 2024-02-10T08:41:44.327968+0000 mgr.y (mgr.14547) 740 : cluster [DBG] pgmap v1223: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:45.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[24488]: audit 2024-02-10T08:41:44.350272+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]: dispatch 2024-02-10T08:41:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[17385]: audit 2024-02-10T08:41:44.055643+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:41:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[17385]: cluster 2024-02-10T08:41:44.055856+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-10T08:41:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[17385]: cluster 2024-02-10T08:41:44.327968+0000 mgr.y (mgr.14547) 740 : cluster [DBG] pgmap v1223: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:45.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:45 smithi067 bash[17385]: audit 2024-02-10T08:41:44.350272+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]: dispatch 2024-02-10T08:41:45.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:45 smithi134 bash[18722]: audit 2024-02-10T08:41:44.055643+0000 mon.a (mon.0) 3136 : audit [INF] from='client.? 172.21.15.67:0/170025336' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:41:45.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:45 smithi134 bash[18722]: cluster 2024-02-10T08:41:44.055856+0000 mon.a (mon.0) 3137 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-10T08:41:45.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:45 smithi134 bash[18722]: cluster 2024-02-10T08:41:44.327968+0000 mgr.y (mgr.14547) 740 : cluster [DBG] pgmap v1223: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:41:45.527 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:45 smithi134 bash[18722]: audit 2024-02-10T08:41:44.350272+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]: dispatch 2024-02-10T08:41:46.089 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.086+0000 7f96ce7fc700 1 -- 172.21.15.67:0/1878446500 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]=0 enabled application 'rados' on pool '34c11e82-d78d-4916-9c14-acce32e59950' v720) v1 ==== 213+0+0 (secure 0 0 0) 0x7f96d404c980 con 0x7f96d80f8750 2024-02-10T08:41:46.092 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f96c4061340 msgr2=0x7f96c4063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:46.092 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f96c4061340 0x7f96c4063800 secure :-1 s=READY pgs=3561 cs=0 l=1 rev1=1 rx=0x7f96d8108ca0 tx=0x7f96c800b040).stop 2024-02-10T08:41:46.092 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 msgr2=0x7f96d810d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d810d7e0 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 rx=0x7f96d400e970 tx=0x7f96d400b040).stop 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 shutdown_connections 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f96c4061340 0x7f96c4063800 unknown :-1 s=CLOSED pgs=3561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d8101cb0 0x7f96d81079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f96d80f90b0 0x7f96d810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 --2- 172.21.15.67:0/1878446500 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f96d80f8750 0x7f96d810d7e0 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 >> 172.21.15.67:0/1878446500 conn(0x7f96d806e610 msgr2=0x7f96d80702b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 shutdown_connections 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:41:46.090+0000 7f96df9dc700 1 -- 172.21.15.67:0/1878446500 wait complete. 2024-02-10T08:41:46.093 INFO:tasks.workunit.client.0.smithi067.stderr:enabled application 'rados' on pool '34c11e82-d78d-4916-9c14-acce32e59950' 2024-02-10T08:41:46.104 INFO:tasks.workunit.client.0.smithi067.stderr:+ seq 1 10 2024-02-10T08:41:46.117 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj1 /etc/passwd 2024-02-10T08:41:46.172 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj2 /etc/passwd 2024-02-10T08:41:46.222 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj3 /etc/passwd 2024-02-10T08:41:46.268 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj4 /etc/passwd 2024-02-10T08:41:46.311 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj5 /etc/passwd 2024-02-10T08:41:46.354 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj6 /etc/passwd 2024-02-10T08:41:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:46 smithi067 bash[24488]: audit 2024-02-10T08:41:45.074596+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]': finished 2024-02-10T08:41:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:46 smithi067 bash[24488]: cluster 2024-02-10T08:41:45.074712+0000 mon.a (mon.0) 3140 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-10T08:41:46.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:46 smithi067 bash[24488]: audit 2024-02-10T08:41:45.178302+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]: dispatch 2024-02-10T08:41:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:46 smithi067 bash[17385]: audit 2024-02-10T08:41:45.074596+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]': finished 2024-02-10T08:41:46.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:46 smithi067 bash[17385]: cluster 2024-02-10T08:41:45.074712+0000 mon.a (mon.0) 3140 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-10T08:41:46.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:46 smithi067 bash[17385]: audit 2024-02-10T08:41:45.178302+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]: dispatch 2024-02-10T08:41:46.403 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj7 /etc/passwd 2024-02-10T08:41:46.445 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj8 /etc/passwd 2024-02-10T08:41:46.487 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj9 /etc/passwd 2024-02-10T08:41:46.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:46 smithi134 bash[18722]: audit 2024-02-10T08:41:45.074596+0000 mon.a (mon.0) 3139 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]': finished 2024-02-10T08:41:46.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:46 smithi134 bash[18722]: cluster 2024-02-10T08:41:45.074712+0000 mon.a (mon.0) 3140 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-10T08:41:46.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:46 smithi134 bash[18722]: audit 2024-02-10T08:41:45.178302+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]: dispatch 2024-02-10T08:41:46.531 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put obj10 /etc/passwd 2024-02-10T08:41:46.575 INFO:tasks.workunit.client.0.smithi067.stderr:+ sleep 30 2024-02-10T08:41:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[17385]: audit 2024-02-10T08:41:46.085225+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]': finished 2024-02-10T08:41:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[17385]: cluster 2024-02-10T08:41:46.085320+0000 mon.a (mon.0) 3143 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-10T08:41:47.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[17385]: cluster 2024-02-10T08:41:46.328548+0000 mgr.y (mgr.14547) 741 : cluster [DBG] pgmap v1226: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:47.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[24488]: audit 2024-02-10T08:41:46.085225+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]': finished 2024-02-10T08:41:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[24488]: cluster 2024-02-10T08:41:46.085320+0000 mon.a (mon.0) 3143 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-10T08:41:47.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[24488]: cluster 2024-02-10T08:41:46.328548+0000 mgr.y (mgr.14547) 741 : cluster [DBG] pgmap v1226: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:47.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:47 smithi134 bash[18722]: audit 2024-02-10T08:41:46.085225+0000 mon.a (mon.0) 3142 : audit [INF] from='client.? 172.21.15.67:0/1878446500' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "app": "rados"}]': finished 2024-02-10T08:41:47.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:47 smithi134 bash[18722]: cluster 2024-02-10T08:41:46.085320+0000 mon.a (mon.0) 3143 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-10T08:41:47.526 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:47 smithi134 bash[18722]: cluster 2024-02-10T08:41:46.328548+0000 mgr.y (mgr.14547) 741 : cluster [DBG] pgmap v1226: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-10T08:41:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:41:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:47] "GET /metrics HTTP/1.1" 200 189422 "" "Prometheus/2.33.4" 2024-02-10T08:41:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:49 smithi134 bash[18722]: cluster 2024-02-10T08:41:48.330329+0000 mgr.y (mgr.14547) 742 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-10T08:41:49.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:49 smithi067 bash[24488]: cluster 2024-02-10T08:41:48.330329+0000 mgr.y (mgr.14547) 742 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-10T08:41:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:49 smithi067 bash[17385]: cluster 2024-02-10T08:41:48.330329+0000 mgr.y (mgr.14547) 742 : cluster [DBG] pgmap v1227: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 970 B/s rd, 23 KiB/s wr, 2 op/s 2024-02-10T08:41:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:51 smithi134 bash[18722]: cluster 2024-02-10T08:41:50.331149+0000 mgr.y (mgr.14547) 743 : cluster [DBG] pgmap v1228: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 815 B/s rd, 19 KiB/s wr, 2 op/s 2024-02-10T08:41:51.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:51 smithi067 bash[17385]: cluster 2024-02-10T08:41:50.331149+0000 mgr.y (mgr.14547) 743 : cluster [DBG] pgmap v1228: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 815 B/s rd, 19 KiB/s wr, 2 op/s 2024-02-10T08:41:51.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:51 smithi067 bash[24488]: cluster 2024-02-10T08:41:50.331149+0000 mgr.y (mgr.14547) 743 : cluster [DBG] pgmap v1228: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 815 B/s rd, 19 KiB/s wr, 2 op/s 2024-02-10T08:41:52.776 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:41:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:41:52.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:52 smithi134 bash[18722]: cluster 2024-02-10T08:41:52.415835+0000 mon.a (mon.0) 3144 : cluster [WRN] pool '34c11e82-d78d-4916-9c14-acce32e59950' is full (reached quota's max_objects: 10) 2024-02-10T08:41:52.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:52 smithi134 bash[18722]: cluster 2024-02-10T08:41:52.416275+0000 mon.a (mon.0) 3145 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:41:52.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:52 smithi134 bash[18722]: cluster 2024-02-10T08:41:52.425214+0000 mon.a (mon.0) 3146 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-10T08:41:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:52 smithi067 bash[17385]: cluster 2024-02-10T08:41:52.415835+0000 mon.a (mon.0) 3144 : cluster [WRN] pool '34c11e82-d78d-4916-9c14-acce32e59950' is full (reached quota's max_objects: 10) 2024-02-10T08:41:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:52 smithi067 bash[17385]: cluster 2024-02-10T08:41:52.416275+0000 mon.a (mon.0) 3145 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:41:52.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:52 smithi067 bash[17385]: cluster 2024-02-10T08:41:52.425214+0000 mon.a (mon.0) 3146 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-10T08:41:52.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:52 smithi067 bash[24488]: cluster 2024-02-10T08:41:52.415835+0000 mon.a (mon.0) 3144 : cluster [WRN] pool '34c11e82-d78d-4916-9c14-acce32e59950' is full (reached quota's max_objects: 10) 2024-02-10T08:41:52.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:52 smithi067 bash[24488]: cluster 2024-02-10T08:41:52.416275+0000 mon.a (mon.0) 3145 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:41:52.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:52 smithi067 bash[24488]: cluster 2024-02-10T08:41:52.425214+0000 mon.a (mon.0) 3146 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-10T08:41:53.438 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:41:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:41:53.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:53 smithi134 bash[18722]: cluster 2024-02-10T08:41:52.332900+0000 mgr.y (mgr.14547) 744 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:41:53.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:53 smithi134 bash[18722]: audit 2024-02-10T08:41:52.553292+0000 mgr.y (mgr.14547) 745 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:53 smithi067 bash[17385]: cluster 2024-02-10T08:41:52.332900+0000 mgr.y (mgr.14547) 744 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:41:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:53 smithi067 bash[17385]: audit 2024-02-10T08:41:52.553292+0000 mgr.y (mgr.14547) 745 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:53.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:53 smithi067 bash[24488]: cluster 2024-02-10T08:41:52.332900+0000 mgr.y (mgr.14547) 744 : cluster [DBG] pgmap v1229: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:41:53.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:53 smithi067 bash[24488]: audit 2024-02-10T08:41:52.553292+0000 mgr.y (mgr.14547) 745 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:41:55.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:55 smithi134 bash[18722]: cluster 2024-02-10T08:41:54.333703+0000 mgr.y (mgr.14547) 746 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:41:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:55 smithi067 bash[17385]: cluster 2024-02-10T08:41:54.333703+0000 mgr.y (mgr.14547) 746 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:41:55.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:55 smithi067 bash[24488]: cluster 2024-02-10T08:41:54.333703+0000 mgr.y (mgr.14547) 746 : cluster [DBG] pgmap v1231: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-10T08:41:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:56 smithi134 bash[18722]: audit 2024-02-10T08:41:55.920344+0000 mon.c (mon.2) 1032 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:41:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:56 smithi134 bash[18722]: audit 2024-02-10T08:41:56.234047+0000 mon.c (mon.2) 1033 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:41:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:56 smithi134 bash[18722]: audit 2024-02-10T08:41:56.235726+0000 mon.c (mon.2) 1034 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:41:56.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:56 smithi134 bash[18722]: audit 2024-02-10T08:41:56.244223+0000 mon.a (mon.0) 3147 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[17385]: audit 2024-02-10T08:41:55.920344+0000 mon.c (mon.2) 1032 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:41:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[17385]: audit 2024-02-10T08:41:56.234047+0000 mon.c (mon.2) 1033 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:41:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[17385]: audit 2024-02-10T08:41:56.235726+0000 mon.c (mon.2) 1034 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:41:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[17385]: audit 2024-02-10T08:41:56.244223+0000 mon.a (mon.0) 3147 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:56.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[24488]: audit 2024-02-10T08:41:55.920344+0000 mon.c (mon.2) 1032 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:41:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[24488]: audit 2024-02-10T08:41:56.234047+0000 mon.c (mon.2) 1033 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:41:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[24488]: audit 2024-02-10T08:41:56.235726+0000 mon.c (mon.2) 1034 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:41:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:56 smithi067 bash[24488]: audit 2024-02-10T08:41:56.244223+0000 mon.a (mon.0) 3147 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:41:57.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:57 smithi134 bash[18722]: cluster 2024-02-10T08:41:56.334504+0000 mgr.y (mgr.14547) 747 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:41:57.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:57 smithi067 bash[17385]: cluster 2024-02-10T08:41:56.334504+0000 mgr.y (mgr.14547) 747 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:41:57.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:57 smithi067 bash[24488]: cluster 2024-02-10T08:41:56.334504+0000 mgr.y (mgr.14547) 747 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:41:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:41:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:41:57] "GET /metrics HTTP/1.1" 200 189422 "" "Prometheus/2.33.4" 2024-02-10T08:41:59.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:41:59 smithi134 bash[18722]: cluster 2024-02-10T08:41:58.336384+0000 mgr.y (mgr.14547) 748 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:41:59.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:41:59 smithi067 bash[17385]: cluster 2024-02-10T08:41:58.336384+0000 mgr.y (mgr.14547) 748 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:41:59.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:41:59 smithi067 bash[24488]: cluster 2024-02-10T08:41:58.336384+0000 mgr.y (mgr.14547) 748 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:01 smithi067 bash[17385]: cluster 2024-02-10T08:42:00.337132+0000 mgr.y (mgr.14547) 749 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:01 smithi067 bash[17385]: audit 2024-02-10T08:42:00.798092+0000 mon.a (mon.0) 3148 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:42:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:01 smithi067 bash[24488]: cluster 2024-02-10T08:42:00.337132+0000 mgr.y (mgr.14547) 749 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:01 smithi067 bash[24488]: audit 2024-02-10T08:42:00.798092+0000 mon.a (mon.0) 3148 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:42:02.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:01 smithi134 bash[18722]: cluster 2024-02-10T08:42:00.337132+0000 mgr.y (mgr.14547) 749 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:02.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:01 smithi134 bash[18722]: audit 2024-02-10T08:42:00.798092+0000 mon.a (mon.0) 3148 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:42:03.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:42:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:42:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:42:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:42:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:03 smithi067 bash[17385]: cluster 2024-02-10T08:42:02.339029+0000 mgr.y (mgr.14547) 750 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:03 smithi067 bash[17385]: audit 2024-02-10T08:42:02.562953+0000 mgr.y (mgr.14547) 751 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:03 smithi067 bash[24488]: cluster 2024-02-10T08:42:02.339029+0000 mgr.y (mgr.14547) 750 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:03 smithi067 bash[24488]: audit 2024-02-10T08:42:02.562953+0000 mgr.y (mgr.14547) 751 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:04.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:03 smithi134 bash[18722]: cluster 2024-02-10T08:42:02.339029+0000 mgr.y (mgr.14547) 750 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:04.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:03 smithi134 bash[18722]: audit 2024-02-10T08:42:02.562953+0000 mgr.y (mgr.14547) 751 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:06.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:05 smithi067 bash[24488]: cluster 2024-02-10T08:42:04.339811+0000 mgr.y (mgr.14547) 752 : cluster [DBG] pgmap v1236: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 859 B/s rd, 0 op/s 2024-02-10T08:42:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:05 smithi067 bash[17385]: cluster 2024-02-10T08:42:04.339811+0000 mgr.y (mgr.14547) 752 : cluster [DBG] pgmap v1236: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 859 B/s rd, 0 op/s 2024-02-10T08:42:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:05 smithi134 bash[18722]: cluster 2024-02-10T08:42:04.339811+0000 mgr.y (mgr.14547) 752 : cluster [DBG] pgmap v1236: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 859 B/s rd, 0 op/s 2024-02-10T08:42:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:07 smithi067 bash[17385]: cluster 2024-02-10T08:42:06.340616+0000 mgr.y (mgr.14547) 753 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:07 smithi067 bash[24488]: cluster 2024-02-10T08:42:06.340616+0000 mgr.y (mgr.14547) 753 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:08.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:42:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:07] "GET /metrics HTTP/1.1" 200 189462 "" "Prometheus/2.33.4" 2024-02-10T08:42:08.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:07 smithi134 bash[18722]: cluster 2024-02-10T08:42:06.340616+0000 mgr.y (mgr.14547) 753 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:10.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:09 smithi067 bash[24488]: cluster 2024-02-10T08:42:08.342470+0000 mgr.y (mgr.14547) 754 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:09 smithi067 bash[17385]: cluster 2024-02-10T08:42:08.342470+0000 mgr.y (mgr.14547) 754 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:10.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:09 smithi134 bash[18722]: cluster 2024-02-10T08:42:08.342470+0000 mgr.y (mgr.14547) 754 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:11 smithi067 bash[17385]: cluster 2024-02-10T08:42:10.343308+0000 mgr.y (mgr.14547) 755 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:12.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:11 smithi067 bash[24488]: cluster 2024-02-10T08:42:10.343308+0000 mgr.y (mgr.14547) 755 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:12.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:11 smithi134 bash[18722]: cluster 2024-02-10T08:42:10.343308+0000 mgr.y (mgr.14547) 755 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:13.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:42:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:42:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:42:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:42:14.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:13 smithi067 bash[24488]: cluster 2024-02-10T08:42:12.345089+0000 mgr.y (mgr.14547) 756 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:14.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:13 smithi067 bash[24488]: audit 2024-02-10T08:42:12.574218+0000 mgr.y (mgr.14547) 757 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:13 smithi067 bash[17385]: cluster 2024-02-10T08:42:12.345089+0000 mgr.y (mgr.14547) 756 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:13 smithi067 bash[17385]: audit 2024-02-10T08:42:12.574218+0000 mgr.y (mgr.14547) 757 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:14.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:13 smithi134 bash[18722]: cluster 2024-02-10T08:42:12.345089+0000 mgr.y (mgr.14547) 756 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:14.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:13 smithi134 bash[18722]: audit 2024-02-10T08:42:12.574218+0000 mgr.y (mgr.14547) 757 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:16.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:15 smithi067 bash[17385]: cluster 2024-02-10T08:42:14.345880+0000 mgr.y (mgr.14547) 758 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:15 smithi067 bash[24488]: cluster 2024-02-10T08:42:14.345880+0000 mgr.y (mgr.14547) 758 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:16.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:15 smithi134 bash[18722]: cluster 2024-02-10T08:42:14.345880+0000 mgr.y (mgr.14547) 758 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:16.576 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=120871 2024-02-10T08:42:16.577 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_objects 100 2024-02-10T08:42:16.577 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put onemore /etc/passwd 2024-02-10T08:42:16.689 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 -- 172.21.15.67:0/2334130240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fd950 msgr2=0x7f35f40fdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:16.689 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 --2- 172.21.15.67:0/2334130240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fd950 0x7f35f40fdd30 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 rx=0x7f35e40048a0 tx=0x7f35e4009cb0).stop 2024-02-10T08:42:16.689 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 -- 172.21.15.67:0/2334130240 shutdown_connections 2024-02-10T08:42:16.690 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 --2- 172.21.15.67:0/2334130240 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f4060c70 0x7f35f4104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:16.690 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 --2- 172.21.15.67:0/2334130240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f40fe2b0 0x7f35f4060590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:16.690 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 --2- 172.21.15.67:0/2334130240 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fd950 0x7f35f40fdd30 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:16.690 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 -- 172.21.15.67:0/2334130240 >> 172.21.15.67:0/2334130240 conn(0x7f35f406c740 msgr2=0x7f35f406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:16.690 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 -- 172.21.15.67:0/2334130240 shutdown_connections 2024-02-10T08:42:16.690 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.686+0000 7f35f9511700 1 -- 172.21.15.67:0/2334130240 wait complete. 2024-02-10T08:42:16.692 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 Processor -- start 2024-02-10T08:42:16.692 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- start start 2024-02-10T08:42:16.692 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4060c70 0x7f35f410db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:16.692 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f40fd950 0x7f35f410e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:16.692 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 0x7f35f4107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41081c0 con 0x7f35f40fe2b0 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f4108330 con 0x7f35f4060c70 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41084f0 con 0x7f35f40fd950 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f27fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f40fd950 0x7f35f410e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 0x7f35f4107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f27fc700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f40fd950 0x7f35f410e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:44928/0 (socket says 172.21.15.67:44928) 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f27fc700 1 -- 172.21.15.67:0/2419573725 learned_addr learned my addr 172.21.15.67:0/2419573725 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 0x7f35f4107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:35498/0 (socket says 172.21.15.67:35498) 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f2ffd700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4060c70 0x7f35f410db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 -- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f40fd950 msgr2=0x7f35f410e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f40fd950 0x7f35f410e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 -- 172.21.15.67:0/2419573725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4060c70 msgr2=0x7f35f410db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4060c70 0x7f35f410db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:16.693 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4005040 con 0x7f35f40fe2b0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f37fe700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 0x7f35f4107cd0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7f35ec00eb70 tx=0x7f35ec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec009870 con 0x7f35f40fe2b0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f35ec00f040 con 0x7f35f40fe2b0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec007c40 con 0x7f35f40fe2b0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f4108770 con 0x7f35f40fe2b0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f4114960 con 0x7f35f40fe2b0 2024-02-10T08:42:16.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.690+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f405f540 con 0x7f35f40fe2b0 2024-02-10T08:42:16.695 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.694+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f35ec0092e0 con 0x7f35f40fe2b0 2024-02-10T08:42:16.695 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.694+0000 7f35dbfff700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f35dc061340 0x7f35dc063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:16.696 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.694+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(721..721 src has 1..721) v4 ==== 8783+0+0 (secure 0 0 0) 0x7f35ec07d210 con 0x7f35f40fe2b0 2024-02-10T08:42:16.696 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.694+0000 7f35f2ffd700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f35dc061340 0x7f35dc063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:16.696 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.694+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=722}) v3 -- 0x7f35dc06c940 con 0x7f35f40fe2b0 2024-02-10T08:42:16.696 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.694+0000 7f35f2ffd700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f35dc061340 0x7f35dc063800 secure :-1 s=READY pgs=3573 cs=0 l=1 rev1=1 rx=0x7f35e4004870 tx=0x7f35e400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:16.703 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.702+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f35ec049f60 con 0x7f35f40fe2b0 2024-02-10T08:42:16.865 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:16.862+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f35f405fae0 con 0x7f35f40fe2b0 2024-02-10T08:42:17.851 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:17.850+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 7 ==== osd_map(722..722 src has 1..722) v4 ==== 620+0+0 (secure 0 0 0) 0x7f35ec009590 con 0x7f35f40fe2b0 2024-02-10T08:42:17.851 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:17.850+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=723}) v3 -- 0x7f35dc06d620 con 0x7f35f40fe2b0 2024-02-10T08:42:17.858 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:17.858+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v722) v1 ==== 225+0+0 (secure 0 0 0) 0x7f35ec04c9e0 con 0x7f35f40fe2b0 2024-02-10T08:42:17.952 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:17.950+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f35f405f020 con 0x7f35f40fe2b0 2024-02-10T08:42:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:17 smithi067 bash[17385]: cluster 2024-02-10T08:42:16.346715+0000 mgr.y (mgr.14547) 759 : cluster [DBG] pgmap v1242: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:17 smithi067 bash[17385]: audit 2024-02-10T08:42:16.865789+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:42:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:42:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:17] "GET /metrics HTTP/1.1" 200 189459 "" "Prometheus/2.33.4" 2024-02-10T08:42:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:17 smithi067 bash[24488]: cluster 2024-02-10T08:42:16.346715+0000 mgr.y (mgr.14547) 759 : cluster [DBG] pgmap v1242: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:17 smithi067 bash[24488]: audit 2024-02-10T08:42:16.865789+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:42:18.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:17 smithi134 bash[18722]: cluster 2024-02-10T08:42:16.346715+0000 mgr.y (mgr.14547) 759 : cluster [DBG] pgmap v1242: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:18.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:17 smithi134 bash[18722]: audit 2024-02-10T08:42:16.865789+0000 mon.a (mon.0) 3149 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:42:18.852 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.849+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 9 ==== osd_map(723..723 src has 1..723) v4 ==== 620+0+0 (secure 0 0 0) 0x7f35ec050640 con 0x7f35f40fe2b0 2024-02-10T08:42:18.852 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.849+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=724}) v3 -- 0x7f35dc06dc80 con 0x7f35f40fe2b0 2024-02-10T08:42:18.861 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.857+0000 7f35dbfff700 1 -- 172.21.15.67:0/2419573725 <== mon.0 v2:172.21.15.67:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v723) v1 ==== 225+0+0 (secure 0 0 0) 0x7f35ec050000 con 0x7f35f40fe2b0 2024-02-10T08:42:18.866 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f35dc061340 msgr2=0x7f35dc063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f35dc061340 0x7f35dc063800 secure :-1 s=READY pgs=3573 cs=0 l=1 rev1=1 rx=0x7f35e4004870 tx=0x7f35e400b040).stop 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 msgr2=0x7f35f4107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 0x7f35f4107cd0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 rx=0x7f35ec00eb70 tx=0x7f35ec00b040).stop 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 shutdown_connections 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f35dc061340 0x7f35dc063800 unknown :-1 s=CLOSED pgs=3573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35f40fe2b0 0x7f35f4107cd0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35f40fd950 0x7f35f410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:18.867 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 --2- 172.21.15.67:0/2419573725 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35f4060c70 0x7f35f410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:18.868 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 >> 172.21.15.67:0/2419573725 conn(0x7f35f406c740 msgr2=0x7f35f40f5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:18.868 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 shutdown_connections 2024-02-10T08:42:18.868 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:18.861+0000 7f35f9511700 1 -- 172.21.15.67:0/2419573725 wait complete. 2024-02-10T08:42:18.868 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_objects = 100 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:42:18.877 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 120871 2024-02-10T08:42:19.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:18 smithi067 bash[17385]: audit 2024-02-10T08:42:17.855228+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:42:19.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:18 smithi067 bash[17385]: cluster 2024-02-10T08:42:17.855515+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-10T08:42:19.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:18 smithi067 bash[17385]: audit 2024-02-10T08:42:17.953184+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:42:19.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:18 smithi067 bash[24488]: audit 2024-02-10T08:42:17.855228+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:42:19.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:18 smithi067 bash[24488]: cluster 2024-02-10T08:42:17.855515+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-10T08:42:19.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:18 smithi067 bash[24488]: audit 2024-02-10T08:42:17.953184+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:42:19.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:18 smithi134 bash[18722]: audit 2024-02-10T08:42:17.855228+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:42:19.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:18 smithi134 bash[18722]: cluster 2024-02-10T08:42:17.855515+0000 mon.a (mon.0) 3151 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-10T08:42:19.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:18 smithi134 bash[18722]: audit 2024-02-10T08:42:17.953184+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]: dispatch 2024-02-10T08:42:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:19 smithi067 bash[17385]: cluster 2024-02-10T08:42:18.347471+0000 mgr.y (mgr.14547) 760 : cluster [DBG] pgmap v1244: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:19 smithi067 bash[17385]: audit 2024-02-10T08:42:18.857908+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:42:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:19 smithi067 bash[17385]: cluster 2024-02-10T08:42:18.858358+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-10T08:42:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:19 smithi067 bash[24488]: cluster 2024-02-10T08:42:18.347471+0000 mgr.y (mgr.14547) 760 : cluster [DBG] pgmap v1244: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:19 smithi067 bash[24488]: audit 2024-02-10T08:42:18.857908+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:42:20.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:19 smithi067 bash[24488]: cluster 2024-02-10T08:42:18.858358+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-10T08:42:20.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:19 smithi134 bash[18722]: cluster 2024-02-10T08:42:18.347471+0000 mgr.y (mgr.14547) 760 : cluster [DBG] pgmap v1244: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:42:20.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:19 smithi134 bash[18722]: audit 2024-02-10T08:42:18.857908+0000 mon.a (mon.0) 3153 : audit [INF] from='client.? 172.21.15.67:0/2419573725' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "100"}]': finished 2024-02-10T08:42:20.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:19 smithi134 bash[18722]: cluster 2024-02-10T08:42:18.858358+0000 mon.a (mon.0) 3154 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-10T08:42:22.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:21 smithi134 bash[18722]: cluster 2024-02-10T08:42:20.348255+0000 mgr.y (mgr.14547) 761 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:42:22.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:21 smithi067 bash[17385]: cluster 2024-02-10T08:42:20.348255+0000 mgr.y (mgr.14547) 761 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:42:22.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:21 smithi067 bash[24488]: cluster 2024-02-10T08:42:20.348255+0000 mgr.y (mgr.14547) 761 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:42:22.447 INFO:tasks.workunit.client.0.smithi067.stderr:+ [ 0 -ne 0 ] 2024-02-10T08:42:22.447 INFO:tasks.workunit.client.0.smithi067.stderr:+ true 2024-02-10T08:42:22.447 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put twomore /etc/passwd 2024-02-10T08:42:22.488 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_bytes 100 2024-02-10T08:42:22.593 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 -- 172.21.15.67:0/1222233946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 msgr2=0x7f7b800fca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 --2- 172.21.15.67:0/1222233946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b800fca60 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 rx=0x7f7b7c004870 tx=0x7f7b7c009cb0).stop 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 -- 172.21.15.67:0/1222233946 shutdown_connections 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 --2- 172.21.15.67:0/1222233946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800f6fc0 0x7f7b801034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 --2- 172.21.15.67:0/1222233946 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800fcf50 0x7f7b800f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 --2- 172.21.15.67:0/1222233946 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b800fca60 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 -- 172.21.15.67:0/1222233946 >> 172.21.15.67:0/1222233946 conn(0x7f7b8006e6d0 msgr2=0x7f7b8006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 -- 172.21.15.67:0/1222233946 shutdown_connections 2024-02-10T08:42:22.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 -- 172.21.15.67:0/1222233946 wait complete. 2024-02-10T08:42:22.596 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.589+0000 7f7b8766d700 1 Processor -- start 2024-02-10T08:42:22.596 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- start start 2024-02-10T08:42:22.596 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 0x7f7b80108b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:22.596 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b80109000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:22.596 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800fcf50 0x7f7b80109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:22.596 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80113d30 con 0x7f7b800fc680 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80113ea0 con 0x7f7b800fcf50 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80114060 con 0x7f7b800f6fc0 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 0x7f7b80108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b84c08700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b80109000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 0x7f7b80108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:44980/0 (socket says 172.21.15.67:44980) 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 -- 172.21.15.67:0/592275114 learned_addr learned my addr 172.21.15.67:0/592275114 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85c0a700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800fcf50 0x7f7b80109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 -- 172.21.15.67:0/592275114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800fcf50 msgr2=0x7f7b80109610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800fcf50 0x7f7b80109610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 -- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 msgr2=0x7f7b80109000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b80109000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 -- 172.21.15.67:0/592275114 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c005040 con 0x7f7b800f6fc0 2024-02-10T08:42:22.597 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85c0a700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800fcf50 0x7f7b80109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b84c08700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b80109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b85409700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 0x7f7b80108b10 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 rx=0x7f7b7c000cc0 tx=0x7f7b7c009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c00e030 con 0x7f7b800f6fc0 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f7b7c004160 con 0x7f7b800f6fc0 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c018070 con 0x7f7b800f6fc0 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b801142e0 con 0x7f7b800f6fc0 2024-02-10T08:42:22.598 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b80114700 con 0x7f7b800f6fc0 2024-02-10T08:42:22.599 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.593+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b80060470 con 0x7f7b800f6fc0 2024-02-10T08:42:22.599 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.597+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7b7c009240 con 0x7f7b800f6fc0 2024-02-10T08:42:22.599 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.597+0000 7f7b727fc700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7b6c061340 0x7f7b6c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:22.599 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.597+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(724..724 src has 1..724) v4 ==== 8783+0+0 (secure 0 0 0) 0x7f7b7c07e610 con 0x7f7b800f6fc0 2024-02-10T08:42:22.599 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.597+0000 7f7b84c08700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7b6c061340 0x7f7b6c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:22.600 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.597+0000 7f7b84c08700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7b6c061340 0x7f7b6c063800 secure :-1 s=READY pgs=3575 cs=0 l=1 rev1=1 rx=0x7f7b800615f0 tx=0x7f7b7400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:22.606 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.601+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f7b7c04df10 con 0x7f7b800f6fc0 2024-02-10T08:42:22.764 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:22.761+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f7b80061330 con 0x7f7b800f6fc0 2024-02-10T08:42:22.884 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:42:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:42:23.147 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:22 smithi134 bash[18722]: cluster 2024-02-10T08:42:22.427150+0000 mon.a (mon.0) 3155 : cluster [INF] pool '34c11e82-d78d-4916-9c14-acce32e59950' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:42:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:22 smithi134 bash[18722]: cluster 2024-02-10T08:42:22.427788+0000 mon.a (mon.0) 3156 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:42:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:22 smithi134 bash[18722]: cluster 2024-02-10T08:42:22.436949+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-10T08:42:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:22 smithi134 bash[18722]: audit 2024-02-10T08:42:22.764776+0000 mon.c (mon.2) 1035 : audit [INF] from='client.? 172.21.15.67:0/592275114' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:23.148 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:22 smithi134 bash[18722]: audit 2024-02-10T08:42:22.765416+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[17385]: cluster 2024-02-10T08:42:22.427150+0000 mon.a (mon.0) 3155 : cluster [INF] pool '34c11e82-d78d-4916-9c14-acce32e59950' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:42:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[17385]: cluster 2024-02-10T08:42:22.427788+0000 mon.a (mon.0) 3156 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:42:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[17385]: cluster 2024-02-10T08:42:22.436949+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-10T08:42:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[17385]: audit 2024-02-10T08:42:22.764776+0000 mon.c (mon.2) 1035 : audit [INF] from='client.? 172.21.15.67:0/592275114' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:23.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[17385]: audit 2024-02-10T08:42:22.765416+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[24488]: cluster 2024-02-10T08:42:22.427150+0000 mon.a (mon.0) 3155 : cluster [INF] pool '34c11e82-d78d-4916-9c14-acce32e59950' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:42:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[24488]: cluster 2024-02-10T08:42:22.427788+0000 mon.a (mon.0) 3156 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:42:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[24488]: cluster 2024-02-10T08:42:22.436949+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-10T08:42:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[24488]: audit 2024-02-10T08:42:22.764776+0000 mon.c (mon.2) 1035 : audit [INF] from='client.? 172.21.15.67:0/592275114' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:23.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:22 smithi067 bash[24488]: audit 2024-02-10T08:42:22.765416+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:23.457 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:23.453+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v725) v1 ==== 221+0+0 (secure 0 0 0) 0x7f7b7c051530 con 0x7f7b800f6fc0 2024-02-10T08:42:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:42:23.553 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:23.549+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f7b80105820 con 0x7f7b800f6fc0 2024-02-10T08:42:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[18722]: cluster 2024-02-10T08:42:22.350307+0000 mgr.y (mgr.14547) 762 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[18722]: audit 2024-02-10T08:42:22.585775+0000 mgr.y (mgr.14547) 763 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:24.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[18722]: audit 2024-02-10T08:42:23.451861+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:42:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[18722]: cluster 2024-02-10T08:42:23.452086+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-10T08:42:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[18722]: audit 2024-02-10T08:42:23.554138+0000 mon.c (mon.2) 1036 : audit [INF] from='client.? 172.21.15.67:0/592275114' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:24.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:23 smithi134 bash[18722]: audit 2024-02-10T08:42:23.554989+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[17385]: cluster 2024-02-10T08:42:22.350307+0000 mgr.y (mgr.14547) 762 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[17385]: audit 2024-02-10T08:42:22.585775+0000 mgr.y (mgr.14547) 763 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[17385]: audit 2024-02-10T08:42:23.451861+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:42:24.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[17385]: cluster 2024-02-10T08:42:23.452086+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[17385]: audit 2024-02-10T08:42:23.554138+0000 mon.c (mon.2) 1036 : audit [INF] from='client.? 172.21.15.67:0/592275114' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[17385]: audit 2024-02-10T08:42:23.554989+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[24488]: cluster 2024-02-10T08:42:22.350307+0000 mgr.y (mgr.14547) 762 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[24488]: audit 2024-02-10T08:42:22.585775+0000 mgr.y (mgr.14547) 763 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[24488]: audit 2024-02-10T08:42:23.451861+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[24488]: cluster 2024-02-10T08:42:23.452086+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[24488]: audit 2024-02-10T08:42:23.554138+0000 mon.c (mon.2) 1036 : audit [INF] from='client.? 172.21.15.67:0/592275114' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:24.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:23 smithi067 bash[24488]: audit 2024-02-10T08:42:23.554989+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-10T08:42:24.458 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.453+0000 7f7b727fc700 1 -- 172.21.15.67:0/592275114 <== mon.2 v2:172.21.15.67:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v726) v1 ==== 221+0+0 (secure 0 0 0) 0x7f7b7c051710 con 0x7f7b800f6fc0 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7b6c061340 msgr2=0x7f7b6c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7b6c061340 0x7f7b6c063800 secure :-1 s=READY pgs=3575 cs=0 l=1 rev1=1 rx=0x7f7b800615f0 tx=0x7f7b7400b040).stop 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 msgr2=0x7f7b80108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 0x7f7b80108b10 secure :-1 s=READY pgs=3056 cs=0 l=1 rev1=1 rx=0x7f7b7c000cc0 tx=0x7f7b7c009cb0).stop 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 shutdown_connections 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f7b6c061340 0x7f7b6c063800 unknown :-1 s=CLOSED pgs=3575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b800fcf50 0x7f7b80109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:24.461 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f7b800fc680 0x7f7b80109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:24.462 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 --2- 172.21.15.67:0/592275114 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f7b800f6fc0 0x7f7b80108b10 unknown :-1 s=CLOSED pgs=3056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:24.462 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 >> 172.21.15.67:0/592275114 conn(0x7f7b8006e6d0 msgr2=0x7f7b800f4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:24.462 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 shutdown_connections 2024-02-10T08:42:24.462 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:24.457+0000 7f7b8766d700 1 -- 172.21.15.67:0/592275114 wait complete. 2024-02-10T08:42:24.462 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_bytes = 100 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:42:24.472 INFO:tasks.workunit.client.0.smithi067.stderr:+ sleep 30 2024-02-10T08:42:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:25 smithi134 bash[18722]: cluster 2024-02-10T08:42:24.351196+0000 mgr.y (mgr.14547) 764 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:25 smithi134 bash[18722]: audit 2024-02-10T08:42:24.455158+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:42:25.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:25 smithi134 bash[18722]: cluster 2024-02-10T08:42:24.455457+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-10T08:42:25.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:25 smithi067 bash[24488]: cluster 2024-02-10T08:42:24.351196+0000 mgr.y (mgr.14547) 764 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:25.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:25 smithi067 bash[24488]: audit 2024-02-10T08:42:24.455158+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:42:25.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:25 smithi067 bash[24488]: cluster 2024-02-10T08:42:24.455457+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-10T08:42:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:25 smithi067 bash[17385]: cluster 2024-02-10T08:42:24.351196+0000 mgr.y (mgr.14547) 764 : cluster [DBG] pgmap v1250: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:25 smithi067 bash[17385]: audit 2024-02-10T08:42:24.455158+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "100"}]': finished 2024-02-10T08:42:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:25 smithi067 bash[17385]: cluster 2024-02-10T08:42:24.455457+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-10T08:42:27.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:27 smithi134 bash[18722]: cluster 2024-02-10T08:42:26.352011+0000 mgr.y (mgr.14547) 765 : cluster [DBG] pgmap v1252: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:27 smithi134 bash[18722]: cluster 2024-02-10T08:42:27.429758+0000 mon.a (mon.0) 3164 : cluster [WRN] pool '34c11e82-d78d-4916-9c14-acce32e59950' is full (reached quota's max_bytes: 100 B) 2024-02-10T08:42:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:27 smithi134 bash[18722]: cluster 2024-02-10T08:42:27.430404+0000 mon.a (mon.0) 3165 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:42:27.777 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:27 smithi134 bash[18722]: cluster 2024-02-10T08:42:27.439132+0000 mon.a (mon.0) 3166 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-10T08:42:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[17385]: cluster 2024-02-10T08:42:26.352011+0000 mgr.y (mgr.14547) 765 : cluster [DBG] pgmap v1252: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[17385]: cluster 2024-02-10T08:42:27.429758+0000 mon.a (mon.0) 3164 : cluster [WRN] pool '34c11e82-d78d-4916-9c14-acce32e59950' is full (reached quota's max_bytes: 100 B) 2024-02-10T08:42:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[17385]: cluster 2024-02-10T08:42:27.430404+0000 mon.a (mon.0) 3165 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:42:27.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[17385]: cluster 2024-02-10T08:42:27.439132+0000 mon.a (mon.0) 3166 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-10T08:42:27.862 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:27] "GET /metrics HTTP/1.1" 200 189459 "" "Prometheus/2.33.4" 2024-02-10T08:42:27.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[24488]: cluster 2024-02-10T08:42:26.352011+0000 mgr.y (mgr.14547) 765 : cluster [DBG] pgmap v1252: 176 pgs: 176 active+clean; 122 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:27.862 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[24488]: cluster 2024-02-10T08:42:27.429758+0000 mon.a (mon.0) 3164 : cluster [WRN] pool '34c11e82-d78d-4916-9c14-acce32e59950' is full (reached quota's max_bytes: 100 B) 2024-02-10T08:42:27.863 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[24488]: cluster 2024-02-10T08:42:27.430404+0000 mon.a (mon.0) 3165 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:42:27.863 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:27 smithi067 bash[24488]: cluster 2024-02-10T08:42:27.439132+0000 mon.a (mon.0) 3166 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-10T08:42:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:29 smithi134 bash[18722]: cluster 2024-02-10T08:42:28.352844+0000 mgr.y (mgr.14547) 766 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 865 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-10T08:42:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:29 smithi067 bash[17385]: cluster 2024-02-10T08:42:28.352844+0000 mgr.y (mgr.14547) 766 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 865 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-10T08:42:29.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:29 smithi067 bash[24488]: cluster 2024-02-10T08:42:28.352844+0000 mgr.y (mgr.14547) 766 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 865 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-10T08:42:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:31 smithi134 bash[18722]: cluster 2024-02-10T08:42:30.353591+0000 mgr.y (mgr.14547) 767 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 742 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-10T08:42:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:31 smithi067 bash[17385]: cluster 2024-02-10T08:42:30.353591+0000 mgr.y (mgr.14547) 767 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 742 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-10T08:42:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:31 smithi067 bash[24488]: cluster 2024-02-10T08:42:30.353591+0000 mgr.y (mgr.14547) 767 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 742 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-10T08:42:33.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:42:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:42:33.473 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:42:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:42:33.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:33 smithi134 bash[18722]: cluster 2024-02-10T08:42:32.355515+0000 mgr.y (mgr.14547) 768 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:42:33.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:33 smithi134 bash[18722]: audit 2024-02-10T08:42:32.595098+0000 mgr.y (mgr.14547) 769 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:33 smithi067 bash[17385]: cluster 2024-02-10T08:42:32.355515+0000 mgr.y (mgr.14547) 768 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:42:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:33 smithi067 bash[17385]: audit 2024-02-10T08:42:32.595098+0000 mgr.y (mgr.14547) 769 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:33.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:33 smithi067 bash[24488]: cluster 2024-02-10T08:42:32.355515+0000 mgr.y (mgr.14547) 768 : cluster [DBG] pgmap v1256: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:42:33.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:33 smithi067 bash[24488]: audit 2024-02-10T08:42:32.595098+0000 mgr.y (mgr.14547) 769 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:35.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:35 smithi134 bash[18722]: cluster 2024-02-10T08:42:34.356316+0000 mgr.y (mgr.14547) 770 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T08:42:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:35 smithi067 bash[17385]: cluster 2024-02-10T08:42:34.356316+0000 mgr.y (mgr.14547) 770 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T08:42:35.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:35 smithi067 bash[24488]: cluster 2024-02-10T08:42:34.356316+0000 mgr.y (mgr.14547) 770 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T08:42:37.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:37 smithi134 bash[18722]: cluster 2024-02-10T08:42:36.357092+0000 mgr.y (mgr.14547) 771 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T08:42:37.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:37 smithi067 bash[17385]: cluster 2024-02-10T08:42:36.357092+0000 mgr.y (mgr.14547) 771 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T08:42:37.861 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:37 smithi067 bash[24488]: cluster 2024-02-10T08:42:36.357092+0000 mgr.y (mgr.14547) 771 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-10T08:42:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:42:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:37] "GET /metrics HTTP/1.1" 200 189467 "" "Prometheus/2.33.4" 2024-02-10T08:42:39.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:39 smithi134 bash[18722]: cluster 2024-02-10T08:42:38.359012+0000 mgr.y (mgr.14547) 772 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-10T08:42:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:39 smithi067 bash[17385]: cluster 2024-02-10T08:42:38.359012+0000 mgr.y (mgr.14547) 772 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-10T08:42:39.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:39 smithi067 bash[24488]: cluster 2024-02-10T08:42:38.359012+0000 mgr.y (mgr.14547) 772 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-10T08:42:41.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:41 smithi134 bash[18722]: cluster 2024-02-10T08:42:40.359805+0000 mgr.y (mgr.14547) 773 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:41.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:41 smithi067 bash[17385]: cluster 2024-02-10T08:42:40.359805+0000 mgr.y (mgr.14547) 773 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:41.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:41 smithi067 bash[24488]: cluster 2024-02-10T08:42:40.359805+0000 mgr.y (mgr.14547) 773 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:43.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:42:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:42:43.503 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:42:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:42:43.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:43 smithi134 bash[18722]: cluster 2024-02-10T08:42:42.361680+0000 mgr.y (mgr.14547) 774 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:43.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:43 smithi134 bash[18722]: audit 2024-02-10T08:42:42.598706+0000 mgr.y (mgr.14547) 775 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:43 smithi067 bash[17385]: cluster 2024-02-10T08:42:42.361680+0000 mgr.y (mgr.14547) 774 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:43.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:43 smithi067 bash[17385]: audit 2024-02-10T08:42:42.598706+0000 mgr.y (mgr.14547) 775 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:43 smithi067 bash[24488]: cluster 2024-02-10T08:42:42.361680+0000 mgr.y (mgr.14547) 774 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:42:43.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:43 smithi067 bash[24488]: audit 2024-02-10T08:42:42.598706+0000 mgr.y (mgr.14547) 775 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:45.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:45 smithi134 bash[18722]: cluster 2024-02-10T08:42:44.362397+0000 mgr.y (mgr.14547) 776 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:45.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:45 smithi067 bash[17385]: cluster 2024-02-10T08:42:44.362397+0000 mgr.y (mgr.14547) 776 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:45.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:45 smithi067 bash[24488]: cluster 2024-02-10T08:42:44.362397+0000 mgr.y (mgr.14547) 776 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:47.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:47 smithi134 bash[18722]: cluster 2024-02-10T08:42:46.363197+0000 mgr.y (mgr.14547) 777 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:47.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:47 smithi067 bash[17385]: cluster 2024-02-10T08:42:46.363197+0000 mgr.y (mgr.14547) 777 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:47.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:47 smithi067 bash[24488]: cluster 2024-02-10T08:42:46.363197+0000 mgr.y (mgr.14547) 777 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:42:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:42:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:47] "GET /metrics HTTP/1.1" 200 189475 "" "Prometheus/2.33.4" 2024-02-10T08:42:49.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:49 smithi134 bash[18722]: cluster 2024-02-10T08:42:48.365036+0000 mgr.y (mgr.14547) 778 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:49.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:49 smithi067 bash[17385]: cluster 2024-02-10T08:42:48.365036+0000 mgr.y (mgr.14547) 778 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:49.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:49 smithi067 bash[24488]: cluster 2024-02-10T08:42:48.365036+0000 mgr.y (mgr.14547) 778 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:51.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:51 smithi134 bash[18722]: cluster 2024-02-10T08:42:50.365858+0000 mgr.y (mgr.14547) 779 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:51.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:51 smithi067 bash[17385]: cluster 2024-02-10T08:42:50.365858+0000 mgr.y (mgr.14547) 779 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:51.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:51 smithi067 bash[24488]: cluster 2024-02-10T08:42:50.365858+0000 mgr.y (mgr.14547) 779 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:53.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:42:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:42:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:42:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:42:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:53 smithi067 bash[17385]: cluster 2024-02-10T08:42:52.367711+0000 mgr.y (mgr.14547) 780 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:53.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:53 smithi067 bash[17385]: audit 2024-02-10T08:42:52.607435+0000 mgr.y (mgr.14547) 781 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:53.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:53 smithi067 bash[24488]: cluster 2024-02-10T08:42:52.367711+0000 mgr.y (mgr.14547) 780 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:53.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:53 smithi067 bash[24488]: audit 2024-02-10T08:42:52.607435+0000 mgr.y (mgr.14547) 781 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:54.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:53 smithi134 bash[18722]: cluster 2024-02-10T08:42:52.367711+0000 mgr.y (mgr.14547) 780 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:54.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:53 smithi134 bash[18722]: audit 2024-02-10T08:42:52.607435+0000 mgr.y (mgr.14547) 781 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:42:54.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ pid=120960 2024-02-10T08:42:54.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_bytes 0 2024-02-10T08:42:54.474 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put two /etc/passwd 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/3405529470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 msgr2=0x7fe18c0f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- 172.21.15.67:0/3405529470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c0f5f10 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 rx=0x7fe17c004870 tx=0x7fe17c009cb0).stop 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/3405529470 shutdown_connections 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- 172.21.15.67:0/3405529470 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0f65f0 0x7fe18c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- 172.21.15.67:0/3405529470 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c0f5f10 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- 172.21.15.67:0/3405529470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe18c0fbd60 0x7fe18c0fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/3405529470 >> 172.21.15.67:0/3405529470 conn(0x7fe18c06db60 msgr2=0x7fe18c06df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/3405529470 shutdown_connections 2024-02-10T08:42:54.587 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/3405529470 wait complete. 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 Processor -- start 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- start start 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe18c0f65f0 0x7fe18c108a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 0x7fe18c108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c109480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c113bb0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c113d20 con 0x7fe18c0f65f0 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c113ee0 con 0x7fe18c0fbd60 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe190f49700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 0x7fe18c108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:54.590 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe190f49700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 0x7fe18c108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:60000/0 (socket says 172.21.15.67:60000) 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe190f49700 1 -- 172.21.15.67:0/241317473 learned_addr learned my addr 172.21.15.67:0/241317473 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe19174a700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe18c0f65f0 0x7fe18c108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 -- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 msgr2=0x7fe18c108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 0x7fe18c108f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 -- 172.21.15.67:0/241317473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe18c0f65f0 msgr2=0x7fe18c108a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe18c0f65f0 0x7fe18c108a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840097c0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe190f49700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 0x7fe18c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe191f4b700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c109480 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 rx=0x7fe184001cb0 tx=0x7fe184004c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840079b0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c005040 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c1145d0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fe1840090d0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.591 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.585+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18400fb40 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.592 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.589+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0606d0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.593 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.589+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fe18401f440 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.593 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.589+0000 7fe1827fc700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fe178061340 0x7fe178063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:54.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.589+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(727..727 src has 1..727) v4 ==== 8783+0+0 (secure 0 0 0) 0x7fe18407dd70 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.589+0000 7fe19174a700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fe178061340 0x7fe178063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:54.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.589+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=728}) v3 -- 0x7fe17806c8c0 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.594 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.593+0000 7fe19174a700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fe178061340 0x7fe178063800 secure :-1 s=READY pgs=3577 cs=0 l=1 rev1=1 rx=0x7fe188004dc0 tx=0x7fe18800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:54.601 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.597+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fe18404d690 con 0x7fe18c0fc6c0 2024-02-10T08:42:54.756 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:54.753+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe18c05f020 con 0x7fe18c0fc6c0 2024-02-10T08:42:55.534 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:55.529+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 7 ==== osd_map(728..728 src has 1..728) v4 ==== 620+0+0 (secure 0 0 0) 0x7fe1840092b0 con 0x7fe18c0fc6c0 2024-02-10T08:42:55.534 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:55.529+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=729}) v3 -- 0x7fe17806d5a0 con 0x7fe18c0fc6c0 2024-02-10T08:42:55.541 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:55.537+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v728) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe184050cb0 con 0x7fe18c0fc6c0 2024-02-10T08:42:55.638 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:55.637+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe18c103770 con 0x7fe18c0fc6c0 2024-02-10T08:42:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:55 smithi067 bash[17385]: cluster 2024-02-10T08:42:54.368534+0000 mgr.y (mgr.14547) 782 : cluster [DBG] pgmap v1267: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:55.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:55 smithi067 bash[17385]: audit 2024-02-10T08:42:54.757082+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:42:55.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:55 smithi067 bash[24488]: cluster 2024-02-10T08:42:54.368534+0000 mgr.y (mgr.14547) 782 : cluster [DBG] pgmap v1267: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:55.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:55 smithi067 bash[24488]: audit 2024-02-10T08:42:54.757082+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:42:56.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:55 smithi134 bash[18722]: cluster 2024-02-10T08:42:54.368534+0000 mgr.y (mgr.14547) 782 : cluster [DBG] pgmap v1267: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-10T08:42:56.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:55 smithi134 bash[18722]: audit 2024-02-10T08:42:54.757082+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:42:56.554 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.549+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 9 ==== osd_map(729..729 src has 1..729) v4 ==== 620+0+0 (secure 0 0 0) 0x7fe1840512f0 con 0x7fe18c0fc6c0 2024-02-10T08:42:56.554 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.549+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=730}) v3 -- 0x7fe17806da70 con 0x7fe18c0fc6c0 2024-02-10T08:42:56.560 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.557+0000 7fe1827fc700 1 -- 172.21.15.67:0/241317473 <== mon.0 v2:172.21.15.67:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v729) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe184050e90 con 0x7fe18c0fc6c0 2024-02-10T08:42:56.564 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fe178061340 msgr2=0x7fe178063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:56.564 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fe178061340 0x7fe178063800 secure :-1 s=READY pgs=3577 cs=0 l=1 rev1=1 rx=0x7fe188004dc0 tx=0x7fe18800b040).stop 2024-02-10T08:42:56.564 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 msgr2=0x7fe18c109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:56.564 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c109480 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 rx=0x7fe184001cb0 tx=0x7fe184004c50).stop 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 shutdown_connections 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fe178061340 0x7fe178063800 unknown :-1 s=CLOSED pgs=3577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fe18c0fc6c0 0x7fe18c109480 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fe18c0fbd60 0x7fe18c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 --2- 172.21.15.67:0/241317473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe18c0f65f0 0x7fe18c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 >> 172.21.15.67:0/241317473 conn(0x7fe18c06db60 msgr2=0x7fe18c0f53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 shutdown_connections 2024-02-10T08:42:56.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.561+0000 7fe1939ae700 1 -- 172.21.15.67:0/241317473 wait complete. 2024-02-10T08:42:56.566 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_bytes = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:42:56.576 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_objects 0 2024-02-10T08:42:56.680 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 -- 172.21.15.67:0/946215934 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 msgr2=0x7f1354102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 --2- 172.21.15.67:0/946215934 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f1354102bf0 secure :-1 s=READY pgs=3059 cs=0 l=1 rev1=1 rx=0x7f134c009be0 tx=0x7f134c004d10).stop 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 -- 172.21.15.67:0/946215934 shutdown_connections 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 --2- 172.21.15.67:0/946215934 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f1354102bf0 unknown :-1 s=CLOSED pgs=3059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 --2- 172.21.15.67:0/946215934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fc6c0 0x7f13540f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 --2- 172.21.15.67:0/946215934 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fbd60 0x7f13540fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 -- 172.21.15.67:0/946215934 >> 172.21.15.67:0/946215934 conn(0x7f135406fc20 msgr2=0x7f13540f3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 -- 172.21.15.67:0/946215934 shutdown_connections 2024-02-10T08:42:56.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.677+0000 7f13595ba700 1 -- 172.21.15.67:0/946215934 wait complete. 2024-02-10T08:42:56.682 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 Processor -- start 2024-02-10T08:42:56.682 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- start start 2024-02-10T08:42:56.684 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f135410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fbd60 0x7f135410bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fc6c0 0x7f13541057a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1354105c90 con 0x7f13540fc6c0 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1354105e00 con 0x7f13540fbd60 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1354105fc0 con 0x7f13540f65f0 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1352ffd700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f135410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1352ffd700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f135410b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:60012/0 (socket says 172.21.15.67:60012) 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fc6c0 0x7f13541057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1352ffd700 1 -- 172.21.15.67:0/627313975 learned_addr learned my addr 172.21.15.67:0/627313975 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f134a7fc700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fbd60 0x7f135410bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 -- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 msgr2=0x7f135410b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f135410b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 -- 172.21.15.67:0/627313975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fbd60 msgr2=0x7f135410bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:56.685 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fbd60 0x7f135410bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134c0097c0 con 0x7f13540fc6c0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1352ffd700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f135410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f134a7fc700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fbd60 0x7f135410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13537fe700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fc6c0 0x7f13541057a0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 rx=0x7f134c009bb0 tx=0x7f134c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c00e040 con 0x7f13540fc6c0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f134c018040 con 0x7f13540fc6c0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1354106240 con 0x7f13540fc6c0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c013c80 con 0x7f13540fc6c0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1354112400 con 0x7f13540fc6c0 2024-02-10T08:42:56.686 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.681+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135405fc90 con 0x7f13540fc6c0 2024-02-10T08:42:56.687 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.685+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f134c013420 con 0x7f13540fc6c0 2024-02-10T08:42:56.688 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.685+0000 7f1350ff9700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1340061370 0x7f1340063830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:56.688 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.685+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(729..729 src has 1..729) v4 ==== 8783+0+0 (secure 0 0 0) 0x7f134c07e430 con 0x7f13540fc6c0 2024-02-10T08:42:56.688 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.685+0000 7f1352ffd700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1340061370 0x7f1340063830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:56.688 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.685+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=730}) v3 -- 0x7f134006c970 con 0x7f13540fc6c0 2024-02-10T08:42:56.688 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.685+0000 7f1352ffd700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1340061370 0x7f1340063830 secure :-1 s=READY pgs=3578 cs=0 l=1 rev1=1 rx=0x7f135406c660 tx=0x7f134400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:56.694 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.689+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f134c021020 con 0x7f13540fc6c0 2024-02-10T08:42:56.852 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:56.849+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f1354106c80 con 0x7f13540fc6c0 2024-02-10T08:42:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[17385]: audit 2024-02-10T08:42:55.540835+0000 mon.a (mon.0) 3168 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:42:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[17385]: cluster 2024-02-10T08:42:55.540990+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-10T08:42:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[17385]: audit 2024-02-10T08:42:55.639356+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:42:56.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[17385]: audit 2024-02-10T08:42:56.272905+0000 mon.c (mon.2) 1037 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:42:56.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[24488]: audit 2024-02-10T08:42:55.540835+0000 mon.a (mon.0) 3168 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:42:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[24488]: cluster 2024-02-10T08:42:55.540990+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-10T08:42:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[24488]: audit 2024-02-10T08:42:55.639356+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:42:56.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:56 smithi067 bash[24488]: audit 2024-02-10T08:42:56.272905+0000 mon.c (mon.2) 1037 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:42:57.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:56 smithi134 bash[18722]: audit 2024-02-10T08:42:55.540835+0000 mon.a (mon.0) 3168 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:42:57.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:56 smithi134 bash[18722]: cluster 2024-02-10T08:42:55.540990+0000 mon.a (mon.0) 3169 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-10T08:42:57.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:56 smithi134 bash[18722]: audit 2024-02-10T08:42:55.639356+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:42:57.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:56 smithi134 bash[18722]: audit 2024-02-10T08:42:56.272905+0000 mon.c (mon.2) 1037 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:42:57.458 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:57.453+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 7 ==== osd_map(730..730 src has 1..730) v4 ==== 620+0+0 (secure 0 0 0) 0x7f134c051860 con 0x7f13540fc6c0 2024-02-10T08:42:57.458 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:57.453+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=731}) v3 -- 0x7f134006d600 con 0x7f13540fc6c0 2024-02-10T08:42:57.462 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:57.457+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v730) v1 ==== 221+0+0 (secure 0 0 0) 0x7f134c021020 con 0x7f13540fc6c0 2024-02-10T08:42:57.566 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:57.565+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f13541047c0 con 0x7f13540fc6c0 2024-02-10T08:42:57.870 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:42:57] "GET /metrics HTTP/1.1" 200 189475 "" "Prometheus/2.33.4" 2024-02-10T08:42:57.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: cluster 2024-02-10T08:42:56.369119+0000 mgr.y (mgr.14547) 783 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: audit 2024-02-10T08:42:56.558866+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: cluster 2024-02-10T08:42:56.558958+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: audit 2024-02-10T08:42:56.583288+0000 mon.c (mon.2) 1038 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: audit 2024-02-10T08:42:56.584395+0000 mon.c (mon.2) 1039 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: audit 2024-02-10T08:42:56.593957+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: audit 2024-02-10T08:42:56.853149+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: cluster 2024-02-10T08:42:57.440512+0000 mon.a (mon.0) 3175 : cluster [INF] pool '34c11e82-d78d-4916-9c14-acce32e59950' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:42:57.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: cluster 2024-02-10T08:42:57.441047+0000 mon.a (mon.0) 3176 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:42:57.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: audit 2024-02-10T08:42:57.461994+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:42:57.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[17385]: cluster 2024-02-10T08:42:57.462235+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-10T08:42:57.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: cluster 2024-02-10T08:42:56.369119+0000 mgr.y (mgr.14547) 783 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-02-10T08:42:57.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: audit 2024-02-10T08:42:56.558866+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:42:57.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: cluster 2024-02-10T08:42:56.558958+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-10T08:42:57.872 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: audit 2024-02-10T08:42:56.583288+0000 mon.c (mon.2) 1038 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: audit 2024-02-10T08:42:56.584395+0000 mon.c (mon.2) 1039 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: audit 2024-02-10T08:42:56.593957+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: audit 2024-02-10T08:42:56.853149+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: cluster 2024-02-10T08:42:57.440512+0000 mon.a (mon.0) 3175 : cluster [INF] pool '34c11e82-d78d-4916-9c14-acce32e59950' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: cluster 2024-02-10T08:42:57.441047+0000 mon.a (mon.0) 3176 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: audit 2024-02-10T08:42:57.461994+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:42:57.873 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:57 smithi067 bash[24488]: cluster 2024-02-10T08:42:57.462235+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: cluster 2024-02-10T08:42:56.369119+0000 mgr.y (mgr.14547) 783 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: audit 2024-02-10T08:42:56.558866+0000 mon.a (mon.0) 3171 : audit [INF] from='client.? 172.21.15.67:0/241317473' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: cluster 2024-02-10T08:42:56.558958+0000 mon.a (mon.0) 3172 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: audit 2024-02-10T08:42:56.583288+0000 mon.c (mon.2) 1038 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: audit 2024-02-10T08:42:56.584395+0000 mon.c (mon.2) 1039 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: audit 2024-02-10T08:42:56.593957+0000 mon.a (mon.0) 3173 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: audit 2024-02-10T08:42:56.853149+0000 mon.a (mon.0) 3174 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:42:58.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: cluster 2024-02-10T08:42:57.440512+0000 mon.a (mon.0) 3175 : cluster [INF] pool '34c11e82-d78d-4916-9c14-acce32e59950' no longer out of quota; removing NO_QUOTA flag 2024-02-10T08:42:58.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: cluster 2024-02-10T08:42:57.441047+0000 mon.a (mon.0) 3176 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-10T08:42:58.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: audit 2024-02-10T08:42:57.461994+0000 mon.a (mon.0) 3177 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:42:58.028 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:57 smithi134 bash[18722]: cluster 2024-02-10T08:42:57.462235+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-10T08:42:58.477 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.473+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 9 ==== osd_map(731..731 src has 1..731) v4 ==== 620+0+0 (secure 0 0 0) 0x7f134c013de0 con 0x7f13540fc6c0 2024-02-10T08:42:58.485 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.481+0000 7f1350ff9700 1 -- 172.21.15.67:0/627313975 <== mon.0 v2:172.21.15.67:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v731) v1 ==== 221+0+0 (secure 0 0 0) 0x7f134c04dc40 con 0x7f13540fc6c0 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1340061370 msgr2=0x7f1340063830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1340061370 0x7f1340063830 secure :-1 s=READY pgs=3578 cs=0 l=1 rev1=1 rx=0x7f135406c660 tx=0x7f134400b040).stop 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fc6c0 msgr2=0x7f13541057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fc6c0 0x7f13541057a0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 rx=0x7f134c009bb0 tx=0x7f134c004d10).stop 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 shutdown_connections 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f1340061370 0x7f1340063830 unknown :-1 s=CLOSED pgs=3578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f13540fc6c0 0x7f13541057a0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.489 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13540fbd60 0x7f135410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.490 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 --2- 172.21.15.67:0/627313975 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f13540f65f0 0x7f135410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.490 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 >> 172.21.15.67:0/627313975 conn(0x7f135406fc20 msgr2=0x7f135406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:58.490 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 shutdown_connections 2024-02-10T08:42:58.490 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.485+0000 7f13595ba700 1 -- 172.21.15.67:0/627313975 wait complete. 2024-02-10T08:42:58.490 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_objects = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:42:58.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ wait 120960 2024-02-10T08:42:58.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ [ 0 -ne 0 ] 2024-02-10T08:42:58.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ true 2024-02-10T08:42:58.500 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put three /etc/passwd 2024-02-10T08:42:58.544 INFO:tasks.workunit.client.0.smithi067.stderr:+ uuidgen 2024-02-10T08:42:58.545 INFO:tasks.workunit.client.0.smithi067.stderr:+ pp=1b373566-6b27-4e28-8afd-e7e8ebc4bc37 2024-02-10T08:42:58.545 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool create 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 12 2024-02-10T08:42:58.643 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- 172.21.15.67:0/1472322088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 msgr2=0x7f6cac0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- 172.21.15.67:0/1472322088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac0f6920 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 rx=0x7f6c9c0047b0 tx=0x7f6c9c009cb0).stop 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- 172.21.15.67:0/1472322088 shutdown_connections 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- 172.21.15.67:0/1472322088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 0x7f6cac106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- 172.21.15.67:0/1472322088 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 0x7f6cac103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- 172.21.15.67:0/1472322088 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac0f6920 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- 172.21.15.67:0/1472322088 >> 172.21.15.67:0/1472322088 conn(0x7f6cac06e640 msgr2=0x7f6cac0f48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- 172.21.15.67:0/1472322088 shutdown_connections 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- 172.21.15.67:0/1472322088 wait complete. 2024-02-10T08:42:58.644 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 Processor -- start 2024-02-10T08:42:58.645 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- start start 2024-02-10T08:42:58.645 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 0x7f6cac110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 0x7f6cac109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac10a210 con 0x7f6cac0f6540 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac10a380 con 0x7f6cac1040a0 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cb213c700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac10a540 con 0x7f6cac0f6e10 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cab7fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac10fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58210/0 (socket says 172.21.15.67:58210) 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cab7fe700 1 -- 172.21.15.67:0/786935777 learned_addr learned my addr 172.21.15.67:0/786935777 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6caaffd700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 0x7f6cac110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:58.646 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.641+0000 7f6cabfff700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 0x7f6cac109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:58.647 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cab7fe700 1 -- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 msgr2=0x7f6cac110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:58.647 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cab7fe700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 0x7f6cac110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.647 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cab7fe700 1 -- 172.21.15.67:0/786935777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 msgr2=0x7f6cac109d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:58.647 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cab7fe700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 0x7f6cac109d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:58.647 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cab7fe700 1 -- 172.21.15.67:0/786935777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca0005040 con 0x7f6cac0f6540 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cabfff700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 0x7f6cac109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6caaffd700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 0x7f6cac110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cab7fe700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac10fb60 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7f6c9c009e70 tx=0x7f6c9c009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c018070 con 0x7f6cac0f6540 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c005040 con 0x7f6cac0f6540 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f6c9c0041c0 con 0x7f6cac0f6540 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c01dbb0 con 0x7f6cac0f6540 2024-02-10T08:42:58.648 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac10aaa0 con 0x7f6cac0f6540 2024-02-10T08:42:58.649 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac060470 con 0x7f6cac0f6540 2024-02-10T08:42:58.650 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.645+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f6c9c009240 con 0x7f6cac0f6540 2024-02-10T08:42:58.651 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.649+0000 7f6ca8ff9700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6c94061340 0x7f6c94063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:58.651 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.649+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(731..731 src has 1..731) v4 ==== 8783+0+0 (secure 0 0 0) 0x7f6c9c07e4a0 con 0x7f6cac0f6540 2024-02-10T08:42:58.651 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.649+0000 7f6caaffd700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6c94061340 0x7f6c94063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:58.651 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.649+0000 7f6caaffd700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6c94061340 0x7f6c94063800 secure :-1 s=READY pgs=3580 cs=0 l=1 rev1=1 rx=0x7f6cac10b4c0 tx=0x7f6ca000b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:58.657 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.653+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f6c9c04dc70 con 0x7f6cac0f6540 2024-02-10T08:42:58.815 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:58.813+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12} v 0) v1 -- 0x7f6cac108bb0 con 0x7f6cac0f6540 2024-02-10T08:42:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:58 smithi067 bash[17385]: audit 2024-02-10T08:42:57.607654+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:42:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:58 smithi067 bash[17385]: audit 2024-02-10T08:42:58.485656+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:42:58.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:58 smithi067 bash[17385]: cluster 2024-02-10T08:42:58.485769+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-10T08:42:58.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:58 smithi067 bash[24488]: audit 2024-02-10T08:42:57.607654+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:42:58.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:58 smithi067 bash[24488]: audit 2024-02-10T08:42:58.485656+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:42:58.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:58 smithi067 bash[24488]: cluster 2024-02-10T08:42:58.485769+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-10T08:42:59.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:58 smithi134 bash[18722]: audit 2024-02-10T08:42:57.607654+0000 mon.a (mon.0) 3179 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:42:59.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:58 smithi134 bash[18722]: audit 2024-02-10T08:42:58.485656+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? 172.21.15.67:0/627313975' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:42:59.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:58 smithi134 bash[18722]: cluster 2024-02-10T08:42:58.485769+0000 mon.a (mon.0) 3181 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-10T08:42:59.643 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.641+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]=0 pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' created v732) v1 ==== 176+0+0 (secure 0 0 0) 0x7f6c9c051290 con 0x7f6cac0f6540 2024-02-10T08:42:59.741 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.737+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12} v 0) v1 -- 0x7f6cac10b200 con 0x7f6cac0f6540 2024-02-10T08:42:59.742 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.737+0000 7f6ca8ff9700 1 -- 172.21.15.67:0/786935777 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]=0 pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' already exists v732) v1 ==== 183+0+0 (secure 0 0 0) 0x7f6c9c051470 con 0x7f6cac0f6540 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6c94061340 msgr2=0x7f6c94063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6c94061340 0x7f6c94063800 secure :-1 s=READY pgs=3580 cs=0 l=1 rev1=1 rx=0x7f6cac10b4c0 tx=0x7f6ca000b040).stop 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 msgr2=0x7f6cac10fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac10fb60 secure :-1 s=READY pgs=2495 cs=0 l=1 rev1=1 rx=0x7f6c9c009e70 tx=0x7f6c9c009cb0).stop 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 shutdown_connections 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f6c94061340 0x7f6c94063800 unknown :-1 s=CLOSED pgs=3580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cac1040a0 0x7f6cac109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.746 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f6cac0f6e10 0x7f6cac110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.747 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 --2- 172.21.15.67:0/786935777 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f6cac0f6540 0x7f6cac10fb60 unknown :-1 s=CLOSED pgs=2495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.747 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 >> 172.21.15.67:0/786935777 conn(0x7f6cac06e640 msgr2=0x7f6cac06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:59.747 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 shutdown_connections 2024-02-10T08:42:59.747 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.741+0000 7f6cb213c700 1 -- 172.21.15.67:0/786935777 wait complete. 2024-02-10T08:42:59.747 INFO:tasks.workunit.client.0.smithi067.stderr:pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' already exists 2024-02-10T08:42:59.764 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool application enable 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 rados 2024-02-10T08:42:59.875 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2290784547 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080fd700 msgr2=0x7f35080fdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:59.875 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2290784547 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080fd700 0x7f35080fdae0 secure :-1 s=READY pgs=3061 cs=0 l=1 rev1=1 rx=0x7f34f8004870 tx=0x7f34f8009cb0).stop 2024-02-10T08:42:59.875 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2290784547 shutdown_connections 2024-02-10T08:42:59.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2290784547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35080f7f40 0x7f3508104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2290784547 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35080fe060 0x7f35080f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2290784547 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080fd700 0x7f35080fdae0 unknown :-1 s=CLOSED pgs=3061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2290784547 >> 172.21.15.67:0/2290784547 conn(0x7f35080f3360 msgr2=0x7f35080f5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:42:59.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2290784547 shutdown_connections 2024-02-10T08:42:59.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2290784547 wait complete. 2024-02-10T08:42:59.877 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 Processor -- start 2024-02-10T08:42:59.877 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- start start 2024-02-10T08:42:59.877 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 0x7f350810da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:59.877 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35080fd700 0x7f350810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:59.877 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35080fe060 0x7f3508107c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:59.877 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508108130 con 0x7f35080fe060 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35081082a0 con 0x7f35080fd700 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3508108460 con 0x7f35080f7f40 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 0x7f350810da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 0x7f350810da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:60064/0 (socket says 172.21.15.67:60064) 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 -- 172.21.15.67:0/2475911643 learned_addr learned my addr 172.21.15.67:0/2475911643 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350cb9b700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35080fe060 0x7f3508107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:59.878 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f35077fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35080fd700 0x7f350810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:59.879 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 -- 172.21.15.67:0/2475911643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35080fd700 msgr2=0x7f350810df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:59.879 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35080fd700 0x7f350810df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.879 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 -- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35080fe060 msgr2=0x7f3508107c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:42:59.879 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35080fe060 0x7f3508107c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:42:59.879 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 -- 172.21.15.67:0/2475911643 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8005040 con 0x7f35080f7f40 2024-02-10T08:42:59.879 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f3507fff700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 0x7f350810da80 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f34f8000e50 tx=0x7f34f8009cb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:59.880 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f800e030 con 0x7f35080f7f40 2024-02-10T08:42:59.880 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f34f80096d0 con 0x7f35080f7f40 2024-02-10T08:42:59.880 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35081086e0 con 0x7f35080f7f40 2024-02-10T08:42:59.880 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8018070 con 0x7f35080f7f40 2024-02-10T08:42:59.880 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.869+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35081148d0 con 0x7f35080f7f40 2024-02-10T08:42:59.881 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.873+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3508060470 con 0x7f35080f7f40 2024-02-10T08:42:59.881 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.873+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f34f8009240 con 0x7f35080f7f40 2024-02-10T08:42:59.882 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.873+0000 7f35057fa700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f34f0061340 0x7f34f0063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:42:59.882 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.873+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(732..732 src has 1..732) v4 ==== 9158+0+0 (secure 0 0 0) 0x7f34f807e670 con 0x7f35080f7f40 2024-02-10T08:42:59.882 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.873+0000 7f35077fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f34f0061340 0x7f34f0063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:42:59.882 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.873+0000 7f35077fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f34f0061340 0x7f34f0063800 secure :-1 s=READY pgs=3581 cs=0 l=1 rev1=1 rx=0x7f35081093e0 tx=0x7f34fc00b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:42:59.883 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:42:59.881+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f34f804dd40 con 0x7f35080f7f40 2024-02-10T08:43:00.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:59 smithi134 bash[18722]: cluster 2024-02-10T08:42:58.369812+0000 mgr.y (mgr.14547) 784 : cluster [DBG] pgmap v1272: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:00.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:42:59 smithi134 bash[18722]: audit 2024-02-10T08:42:58.815889+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]: dispatch 2024-02-10T08:43:00.038 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:00.033+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"} v 0) v1 -- 0x7f35081069e0 con 0x7f35080f7f40 2024-02-10T08:43:00.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:59 smithi067 bash[17385]: cluster 2024-02-10T08:42:58.369812+0000 mgr.y (mgr.14547) 784 : cluster [DBG] pgmap v1272: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:00.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:42:59 smithi067 bash[17385]: audit 2024-02-10T08:42:58.815889+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]: dispatch 2024-02-10T08:43:00.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:59 smithi067 bash[24488]: cluster 2024-02-10T08:42:58.369812+0000 mgr.y (mgr.14547) 784 : cluster [DBG] pgmap v1272: 176 pgs: 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:00.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:42:59 smithi067 bash[24488]: audit 2024-02-10T08:42:58.815889+0000 mon.a (mon.0) 3182 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]: dispatch 2024-02-10T08:43:00.655 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:00.653+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]=0 enabled application 'rados' on pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' v733) v1 ==== 213+0+0 (secure 0 0 0) 0x7f34f8051360 con 0x7f35080f7f40 2024-02-10T08:43:00.756 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:00.753+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"} v 0) v1 -- 0x7f3508109120 con 0x7f35080f7f40 2024-02-10T08:43:01.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:00 smithi134 bash[18722]: audit 2024-02-10T08:42:59.642883+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]': finished 2024-02-10T08:43:01.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:00 smithi134 bash[18722]: cluster 2024-02-10T08:42:59.643046+0000 mon.a (mon.0) 3184 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-10T08:43:01.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:00 smithi134 bash[18722]: audit 2024-02-10T08:42:59.742132+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]: dispatch 2024-02-10T08:43:01.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:00 smithi134 bash[18722]: audit 2024-02-10T08:43:00.038931+0000 mon.c (mon.2) 1040 : audit [INF] from='client.? 172.21.15.67:0/2475911643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:01.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:00 smithi134 bash[18722]: audit 2024-02-10T08:43:00.039683+0000 mon.a (mon.0) 3186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[17385]: audit 2024-02-10T08:42:59.642883+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]': finished 2024-02-10T08:43:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[17385]: cluster 2024-02-10T08:42:59.643046+0000 mon.a (mon.0) 3184 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-10T08:43:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[17385]: audit 2024-02-10T08:42:59.742132+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]: dispatch 2024-02-10T08:43:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[17385]: audit 2024-02-10T08:43:00.038931+0000 mon.c (mon.2) 1040 : audit [INF] from='client.? 172.21.15.67:0/2475911643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:01.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[17385]: audit 2024-02-10T08:43:00.039683+0000 mon.a (mon.0) 3186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[24488]: audit 2024-02-10T08:42:59.642883+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]': finished 2024-02-10T08:43:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[24488]: cluster 2024-02-10T08:42:59.643046+0000 mon.a (mon.0) 3184 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-10T08:43:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[24488]: audit 2024-02-10T08:42:59.742132+0000 mon.a (mon.0) 3185 : audit [INF] from='client.? 172.21.15.67:0/786935777' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pg_num": 12}]: dispatch 2024-02-10T08:43:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[24488]: audit 2024-02-10T08:43:00.038931+0000 mon.c (mon.2) 1040 : audit [INF] from='client.? 172.21.15.67:0/2475911643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:01.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:00 smithi067 bash[24488]: audit 2024-02-10T08:43:00.039683+0000 mon.a (mon.0) 3186 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:01.655 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.653+0000 7f35057fa700 1 -- 172.21.15.67:0/2475911643 <== mon.2 v2:172.21.15.67:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]=0 enabled application 'rados' on pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' v734) v1 ==== 213+0+0 (secure 0 0 0) 0x7f34f8051540 con 0x7f35080f7f40 2024-02-10T08:43:01.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f34f0061340 msgr2=0x7f34f0063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:01.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f34f0061340 0x7f34f0063800 secure :-1 s=READY pgs=3581 cs=0 l=1 rev1=1 rx=0x7f35081093e0 tx=0x7f34fc00b040).stop 2024-02-10T08:43:01.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 msgr2=0x7f350810da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:01.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 0x7f350810da80 secure :-1 s=READY pgs=3062 cs=0 l=1 rev1=1 rx=0x7f34f8000e50 tx=0x7f34f8009cb0).stop 2024-02-10T08:43:01.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 shutdown_connections 2024-02-10T08:43:01.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f34f0061340 0x7f34f0063800 secure :-1 s=CLOSED pgs=3581 cs=0 l=1 rev1=1 rx=0x7f35081093e0 tx=0x7f34fc00b040).stop 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f35080fe060 0x7f3508107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f35080fd700 0x7f350810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 --2- 172.21.15.67:0/2475911643 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f35080f7f40 0x7f350810da80 unknown :-1 s=CLOSED pgs=3062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 >> 172.21.15.67:0/2475911643 conn(0x7f35080f3360 msgr2=0x7f35080f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 shutdown_connections 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.657+0000 7f350e5fe700 1 -- 172.21.15.67:0/2475911643 wait complete. 2024-02-10T08:43:01.660 INFO:tasks.workunit.client.0.smithi067.stderr:enabled application 'rados' on pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' 2024-02-10T08:43:01.670 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 max_objects 10 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 -- 172.21.15.67:0/679531230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00fdfb0 msgr2=0x7f43c00f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 --2- 172.21.15.67:0/679531230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00fdfb0 0x7f43c00f5c50 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 rx=0x7f43b00047b0 tx=0x7f43b0009cb0).stop 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 -- 172.21.15.67:0/679531230 shutdown_connections 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 --2- 172.21.15.67:0/679531230 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43c00f6220 0x7f43c00f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 --2- 172.21.15.67:0/679531230 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00fdfb0 0x7f43c00f5c50 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 --2- 172.21.15.67:0/679531230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 0x7f43c00fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.777 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 -- 172.21.15.67:0/679531230 >> 172.21.15.67:0/679531230 conn(0x7f43c006fc20 msgr2=0x7f43c00f3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:43:01.778 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 -- 172.21.15.67:0/679531230 shutdown_connections 2024-02-10T08:43:01.778 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.773+0000 7f43c70d0700 1 -- 172.21.15.67:0/679531230 wait complete. 2024-02-10T08:43:01.778 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 Processor -- start 2024-02-10T08:43:01.778 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- start start 2024-02-10T08:43:01.780 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 0x7f43c010dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:43:01.780 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 0x7f43c010e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43c00fdfb0 0x7f43c0107d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0108270 con 0x7f43c00f6220 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01083e0 con 0x7f43c00fd6e0 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01085a0 con 0x7f43c00fdfb0 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 0x7f43c010dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 0x7f43c010dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:58260/0 (socket says 172.21.15.67:58260) 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 -- 172.21.15.67:0/1447572967 learned_addr learned my addr 172.21.15.67:0/1447572967 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c566d700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43c00fdfb0 0x7f43c0107d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43bffff700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 0x7f43c010e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 -- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43c00fdfb0 msgr2=0x7f43c0107d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43c00fdfb0 0x7f43c0107d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 -- 172.21.15.67:0/1447572967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 msgr2=0x7f43c010e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:01.781 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 0x7f43c010e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 -- 172.21.15.67:0/1447572967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0005040 con 0x7f43c00f6220 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43bffff700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 0x7f43c010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c4e6c700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 0x7f43c010dbc0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7f43a800e940 tx=0x7f43a800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8004660 con 0x7f43c00f6220 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f43a800f040 con 0x7f43c00f6220 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c0108880 con 0x7f43c00f6220 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a80146f0 con 0x7f43c00f6220 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c0114a10 con 0x7f43c00f6220 2024-02-10T08:43:01.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.777+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c005fc90 con 0x7f43c00f6220 2024-02-10T08:43:01.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.781+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f43a8007b40 con 0x7f43c00f6220 2024-02-10T08:43:01.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.781+0000 7f43bdffb700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f43ac061340 0x7f43ac063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:43:01.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.781+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(734..734 src has 1..734) v4 ==== 9171+0+0 (secure 0 0 0) 0x7f43a807d240 con 0x7f43c00f6220 2024-02-10T08:43:01.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.781+0000 7f43bffff700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f43ac061340 0x7f43ac063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:43:01.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.781+0000 7f43bffff700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f43ac061340 0x7f43ac063800 secure :-1 s=READY pgs=3582 cs=0 l=1 rev1=1 rx=0x7f43c0108e70 tx=0x7f43b0009cb0).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:43:01.791 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.789+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f43a804c850 con 0x7f43c00f6220 2024-02-10T08:43:01.957 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:01.953+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f43c0106960 con 0x7f43c00f6220 2024-02-10T08:43:02.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:01 smithi134 bash[18722]: cluster 2024-02-10T08:43:00.370636+0000 mgr.y (mgr.14547) 785 : cluster [DBG] pgmap v1275: 188 pgs: 12 unknown, 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:02.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:01 smithi134 bash[18722]: audit 2024-02-10T08:43:00.650829+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]': finished 2024-02-10T08:43:02.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:01 smithi134 bash[18722]: cluster 2024-02-10T08:43:00.650996+0000 mon.a (mon.0) 3188 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-10T08:43:02.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:01 smithi134 bash[18722]: audit 2024-02-10T08:43:00.757090+0000 mon.c (mon.2) 1041 : audit [INF] from='client.? 172.21.15.67:0/2475911643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:02.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:01 smithi134 bash[18722]: audit 2024-02-10T08:43:00.758069+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:02.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:01 smithi134 bash[18722]: audit 2024-02-10T08:43:00.794897+0000 mon.a (mon.0) 3190 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[17385]: cluster 2024-02-10T08:43:00.370636+0000 mgr.y (mgr.14547) 785 : cluster [DBG] pgmap v1275: 188 pgs: 12 unknown, 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[17385]: audit 2024-02-10T08:43:00.650829+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]': finished 2024-02-10T08:43:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[17385]: cluster 2024-02-10T08:43:00.650996+0000 mon.a (mon.0) 3188 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-10T08:43:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[17385]: audit 2024-02-10T08:43:00.757090+0000 mon.c (mon.2) 1041 : audit [INF] from='client.? 172.21.15.67:0/2475911643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[17385]: audit 2024-02-10T08:43:00.758069+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[17385]: audit 2024-02-10T08:43:00.794897+0000 mon.a (mon.0) 3190 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[24488]: cluster 2024-02-10T08:43:00.370636+0000 mgr.y (mgr.14547) 785 : cluster [DBG] pgmap v1275: 188 pgs: 12 unknown, 176 active+clean; 146 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[24488]: audit 2024-02-10T08:43:00.650829+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]': finished 2024-02-10T08:43:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[24488]: cluster 2024-02-10T08:43:00.650996+0000 mon.a (mon.0) 3188 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-10T08:43:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[24488]: audit 2024-02-10T08:43:00.757090+0000 mon.c (mon.2) 1041 : audit [INF] from='client.? 172.21.15.67:0/2475911643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[24488]: audit 2024-02-10T08:43:00.758069+0000 mon.a (mon.0) 3189 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]: dispatch 2024-02-10T08:43:02.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:01 smithi067 bash[24488]: audit 2024-02-10T08:43:00.794897+0000 mon.a (mon.0) 3190 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:02.670 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:02.665+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 v735) v1 ==== 223+0+0 (secure 0 0 0) 0x7f43c0106960 con 0x7f43c00f6220 2024-02-10T08:43:02.769 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:02.765+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f43c0106960 con 0x7f43c00f6220 2024-02-10T08:43:03.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:43:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:43:03.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:02 smithi134 bash[18722]: audit 2024-02-10T08:43:01.650637+0000 mon.a (mon.0) 3191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]': finished 2024-02-10T08:43:03.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:02 smithi134 bash[18722]: cluster 2024-02-10T08:43:01.650764+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-10T08:43:03.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:02 smithi134 bash[18722]: audit 2024-02-10T08:43:01.958106+0000 mon.a (mon.0) 3193 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:43:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:02 smithi067 bash[17385]: audit 2024-02-10T08:43:01.650637+0000 mon.a (mon.0) 3191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]': finished 2024-02-10T08:43:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:02 smithi067 bash[17385]: cluster 2024-02-10T08:43:01.650764+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-10T08:43:03.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:02 smithi067 bash[17385]: audit 2024-02-10T08:43:01.958106+0000 mon.a (mon.0) 3193 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:43:03.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:02 smithi067 bash[24488]: audit 2024-02-10T08:43:01.650637+0000 mon.a (mon.0) 3191 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "app": "rados"}]': finished 2024-02-10T08:43:03.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:02 smithi067 bash[24488]: cluster 2024-02-10T08:43:01.650764+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-10T08:43:03.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:02 smithi067 bash[24488]: audit 2024-02-10T08:43:01.958106+0000 mon.a (mon.0) 3193 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:43:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:43:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:43:03.676 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.673+0000 7f43bdffb700 1 -- 172.21.15.67:0/1447572967 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 v736) v1 ==== 223+0+0 (secure 0 0 0) 0x7f43a8019020 con 0x7f43c00f6220 2024-02-10T08:43:03.680 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f43ac061340 msgr2=0x7f43ac063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:03.680 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f43ac061340 0x7f43ac063800 secure :-1 s=READY pgs=3582 cs=0 l=1 rev1=1 rx=0x7f43c0108e70 tx=0x7f43b0009cb0).stop 2024-02-10T08:43:03.680 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 msgr2=0x7f43c010dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:43:03.680 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 0x7f43c010dbc0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 rx=0x7f43a800e940 tx=0x7f43a800b040).stop 2024-02-10T08:43:03.680 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 shutdown_connections 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f43ac061340 0x7f43ac063800 unknown :-1 s=CLOSED pgs=3582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f43c00fdfb0 0x7f43c0107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43c00fd6e0 0x7f43c010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 --2- 172.21.15.67:0/1447572967 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f43c00f6220 0x7f43c010dbc0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 >> 172.21.15.67:0/1447572967 conn(0x7f43c006fc20 msgr2=0x7f43c006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 shutdown_connections 2024-02-10T08:43:03.681 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:43:03.677+0000 7f43c70d0700 1 -- 172.21.15.67:0/1447572967 wait complete. 2024-02-10T08:43:03.682 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_objects = 10 for pool 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 2024-02-10T08:43:03.692 INFO:tasks.workunit.client.0.smithi067.stderr:+ sleep 30 2024-02-10T08:43:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:03 smithi134 bash[18722]: cluster 2024-02-10T08:43:02.372737+0000 mgr.y (mgr.14547) 786 : cluster [DBG] pgmap v1278: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T08:43:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:03 smithi134 bash[18722]: audit 2024-02-10T08:43:02.618925+0000 mgr.y (mgr.14547) 787 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:03 smithi134 bash[18722]: audit 2024-02-10T08:43:02.669868+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:43:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:03 smithi134 bash[18722]: cluster 2024-02-10T08:43:02.669946+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-10T08:43:04.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:03 smithi134 bash[18722]: audit 2024-02-10T08:43:02.769949+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:43:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[17385]: cluster 2024-02-10T08:43:02.372737+0000 mgr.y (mgr.14547) 786 : cluster [DBG] pgmap v1278: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T08:43:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[17385]: audit 2024-02-10T08:43:02.618925+0000 mgr.y (mgr.14547) 787 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[17385]: audit 2024-02-10T08:43:02.669868+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:43:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[17385]: cluster 2024-02-10T08:43:02.669946+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-10T08:43:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[17385]: audit 2024-02-10T08:43:02.769949+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:43:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[24488]: cluster 2024-02-10T08:43:02.372737+0000 mgr.y (mgr.14547) 786 : cluster [DBG] pgmap v1278: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T08:43:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[24488]: audit 2024-02-10T08:43:02.618925+0000 mgr.y (mgr.14547) 787 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[24488]: audit 2024-02-10T08:43:02.669868+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:43:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[24488]: cluster 2024-02-10T08:43:02.669946+0000 mon.a (mon.0) 3195 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-10T08:43:04.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:03 smithi067 bash[24488]: audit 2024-02-10T08:43:02.769949+0000 mon.a (mon.0) 3196 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]: dispatch 2024-02-10T08:43:05.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:04 smithi134 bash[18722]: audit 2024-02-10T08:43:03.675995+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:43:05.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:04 smithi134 bash[18722]: cluster 2024-02-10T08:43:03.676230+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-10T08:43:05.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:04 smithi067 bash[24488]: audit 2024-02-10T08:43:03.675995+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:43:05.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:04 smithi067 bash[24488]: cluster 2024-02-10T08:43:03.676230+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-10T08:43:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:04 smithi067 bash[17385]: audit 2024-02-10T08:43:03.675995+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? 172.21.15.67:0/1447572967' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "field": "max_objects", "val": "10"}]': finished 2024-02-10T08:43:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:04 smithi067 bash[17385]: cluster 2024-02-10T08:43:03.676230+0000 mon.a (mon.0) 3198 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-10T08:43:06.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:05 smithi134 bash[18722]: cluster 2024-02-10T08:43:04.373604+0000 mgr.y (mgr.14547) 788 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T08:43:06.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:05 smithi067 bash[17385]: cluster 2024-02-10T08:43:04.373604+0000 mgr.y (mgr.14547) 788 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T08:43:06.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:05 smithi067 bash[24488]: cluster 2024-02-10T08:43:04.373604+0000 mgr.y (mgr.14547) 788 : cluster [DBG] pgmap v1281: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-10T08:43:08.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:07 smithi134 bash[18722]: cluster 2024-02-10T08:43:06.374433+0000 mgr.y (mgr.14547) 789 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 894 B/s rd, 4.2 KiB/s wr, 1 op/s 2024-02-10T08:43:08.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:07 smithi067 bash[17385]: cluster 2024-02-10T08:43:06.374433+0000 mgr.y (mgr.14547) 789 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 894 B/s rd, 4.2 KiB/s wr, 1 op/s 2024-02-10T08:43:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:07 smithi067 bash[24488]: cluster 2024-02-10T08:43:06.374433+0000 mgr.y (mgr.14547) 789 : cluster [DBG] pgmap v1282: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 894 B/s rd, 4.2 KiB/s wr, 1 op/s 2024-02-10T08:43:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:43:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:07] "GET /metrics HTTP/1.1" 200 192022 "" "Prometheus/2.33.4" 2024-02-10T08:43:10.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:09 smithi134 bash[18722]: cluster 2024-02-10T08:43:08.376439+0000 mgr.y (mgr.14547) 790 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-02-10T08:43:10.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:09 smithi067 bash[17385]: cluster 2024-02-10T08:43:08.376439+0000 mgr.y (mgr.14547) 790 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-02-10T08:43:10.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:09 smithi067 bash[24488]: cluster 2024-02-10T08:43:08.376439+0000 mgr.y (mgr.14547) 790 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-02-10T08:43:12.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:11 smithi134 bash[18722]: cluster 2024-02-10T08:43:10.377279+0000 mgr.y (mgr.14547) 791 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:43:12.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:11 smithi067 bash[17385]: cluster 2024-02-10T08:43:10.377279+0000 mgr.y (mgr.14547) 791 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:43:12.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:11 smithi067 bash[24488]: cluster 2024-02-10T08:43:10.377279+0000 mgr.y (mgr.14547) 791 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:43:13.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:43:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:43:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:43:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:43:14.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:13 smithi134 bash[18722]: cluster 2024-02-10T08:43:12.379316+0000 mgr.y (mgr.14547) 792 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-10T08:43:14.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:13 smithi134 bash[18722]: audit 2024-02-10T08:43:12.630247+0000 mgr.y (mgr.14547) 793 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:13 smithi067 bash[17385]: cluster 2024-02-10T08:43:12.379316+0000 mgr.y (mgr.14547) 792 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-10T08:43:14.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:13 smithi067 bash[17385]: audit 2024-02-10T08:43:12.630247+0000 mgr.y (mgr.14547) 793 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:14.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:13 smithi067 bash[24488]: cluster 2024-02-10T08:43:12.379316+0000 mgr.y (mgr.14547) 792 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-10T08:43:14.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:13 smithi067 bash[24488]: audit 2024-02-10T08:43:12.630247+0000 mgr.y (mgr.14547) 793 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:16.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:15 smithi134 bash[18722]: cluster 2024-02-10T08:43:14.380125+0000 mgr.y (mgr.14547) 794 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-02-10T08:43:16.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:15 smithi067 bash[17385]: cluster 2024-02-10T08:43:14.380125+0000 mgr.y (mgr.14547) 794 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-02-10T08:43:16.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:15 smithi067 bash[24488]: cluster 2024-02-10T08:43:14.380125+0000 mgr.y (mgr.14547) 794 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 956 B/s rd, 0 op/s 2024-02-10T08:43:18.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:17 smithi134 bash[18722]: cluster 2024-02-10T08:43:16.380945+0000 mgr.y (mgr.14547) 795 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:18.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:17 smithi067 bash[17385]: cluster 2024-02-10T08:43:16.380945+0000 mgr.y (mgr.14547) 795 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:43:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:17] "GET /metrics HTTP/1.1" 200 192024 "" "Prometheus/2.33.4" 2024-02-10T08:43:18.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:17 smithi067 bash[24488]: cluster 2024-02-10T08:43:16.380945+0000 mgr.y (mgr.14547) 795 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:20.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:19 smithi134 bash[18722]: cluster 2024-02-10T08:43:18.382988+0000 mgr.y (mgr.14547) 796 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:20.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:19 smithi067 bash[17385]: cluster 2024-02-10T08:43:18.382988+0000 mgr.y (mgr.14547) 796 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:20.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:19 smithi067 bash[24488]: cluster 2024-02-10T08:43:18.382988+0000 mgr.y (mgr.14547) 796 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:22.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:21 smithi134 bash[18722]: cluster 2024-02-10T08:43:20.383793+0000 mgr.y (mgr.14547) 797 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:22.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:21 smithi067 bash[17385]: cluster 2024-02-10T08:43:20.383793+0000 mgr.y (mgr.14547) 797 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:22.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:21 smithi067 bash[24488]: cluster 2024-02-10T08:43:20.383793+0000 mgr.y (mgr.14547) 797 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:23.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:43:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:43:23.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:43:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:43:24.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:23 smithi134 bash[18722]: cluster 2024-02-10T08:43:22.385714+0000 mgr.y (mgr.14547) 798 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:24.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:23 smithi134 bash[18722]: audit 2024-02-10T08:43:22.640752+0000 mgr.y (mgr.14547) 799 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:23 smithi067 bash[24488]: cluster 2024-02-10T08:43:22.385714+0000 mgr.y (mgr.14547) 798 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:24.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:23 smithi067 bash[24488]: audit 2024-02-10T08:43:22.640752+0000 mgr.y (mgr.14547) 799 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:24.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:23 smithi067 bash[17385]: cluster 2024-02-10T08:43:22.385714+0000 mgr.y (mgr.14547) 798 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:24.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:23 smithi067 bash[17385]: audit 2024-02-10T08:43:22.640752+0000 mgr.y (mgr.14547) 799 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:26.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:25 smithi134 bash[18722]: cluster 2024-02-10T08:43:24.386496+0000 mgr.y (mgr.14547) 800 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:26.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:25 smithi067 bash[17385]: cluster 2024-02-10T08:43:24.386496+0000 mgr.y (mgr.14547) 800 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:26.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:25 smithi067 bash[24488]: cluster 2024-02-10T08:43:24.386496+0000 mgr.y (mgr.14547) 800 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:28.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:27 smithi134 bash[18722]: cluster 2024-02-10T08:43:26.387321+0000 mgr.y (mgr.14547) 801 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:28.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:27 smithi067 bash[17385]: cluster 2024-02-10T08:43:26.387321+0000 mgr.y (mgr.14547) 801 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:43:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:27] "GET /metrics HTTP/1.1" 200 192024 "" "Prometheus/2.33.4" 2024-02-10T08:43:28.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:27 smithi067 bash[24488]: cluster 2024-02-10T08:43:26.387321+0000 mgr.y (mgr.14547) 801 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:30.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:29 smithi134 bash[18722]: cluster 2024-02-10T08:43:28.389321+0000 mgr.y (mgr.14547) 802 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:30.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:29 smithi067 bash[17385]: cluster 2024-02-10T08:43:28.389321+0000 mgr.y (mgr.14547) 802 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:30.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:29 smithi067 bash[24488]: cluster 2024-02-10T08:43:28.389321+0000 mgr.y (mgr.14547) 802 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:32.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:31 smithi134 bash[18722]: cluster 2024-02-10T08:43:30.390126+0000 mgr.y (mgr.14547) 803 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:32.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:31 smithi067 bash[17385]: cluster 2024-02-10T08:43:30.390126+0000 mgr.y (mgr.14547) 803 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:32.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:31 smithi067 bash[24488]: cluster 2024-02-10T08:43:30.390126+0000 mgr.y (mgr.14547) 803 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:33.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:43:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:43:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:43:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:43:33.694 INFO:tasks.workunit.client.0.smithi067.stderr:+ seq 1 10 2024-02-10T08:43:33.695 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj1 /etc/passwd 2024-02-10T08:43:33.742 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj2 /etc/passwd 2024-02-10T08:43:33.789 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj3 /etc/passwd 2024-02-10T08:43:33.832 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj4 /etc/passwd 2024-02-10T08:43:33.874 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj5 /etc/passwd 2024-02-10T08:43:33.917 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj6 /etc/passwd 2024-02-10T08:43:33.960 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj7 /etc/passwd 2024-02-10T08:43:34.002 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj8 /etc/passwd 2024-02-10T08:43:34.046 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj9 /etc/passwd 2024-02-10T08:43:34.088 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 put obj10 /etc/passwd 2024-02-10T08:43:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:33 smithi067 bash[17385]: cluster 2024-02-10T08:43:32.392177+0000 mgr.y (mgr.14547) 804 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:34.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:33 smithi067 bash[17385]: audit 2024-02-10T08:43:32.646279+0000 mgr.y (mgr.14547) 805 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:34.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:33 smithi067 bash[24488]: cluster 2024-02-10T08:43:32.392177+0000 mgr.y (mgr.14547) 804 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:34.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:33 smithi067 bash[24488]: audit 2024-02-10T08:43:32.646279+0000 mgr.y (mgr.14547) 805 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:34.131 INFO:tasks.workunit.client.0.smithi067.stderr:+ sleep 30 2024-02-10T08:43:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:33 smithi134 bash[18722]: cluster 2024-02-10T08:43:32.392177+0000 mgr.y (mgr.14547) 804 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:43:34.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:33 smithi134 bash[18722]: audit 2024-02-10T08:43:32.646279+0000 mgr.y (mgr.14547) 805 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:36.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:35 smithi067 bash[17385]: cluster 2024-02-10T08:43:34.393098+0000 mgr.y (mgr.14547) 806 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:36.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:35 smithi067 bash[24488]: cluster 2024-02-10T08:43:34.393098+0000 mgr.y (mgr.14547) 806 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:36.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:35 smithi134 bash[18722]: cluster 2024-02-10T08:43:34.393098+0000 mgr.y (mgr.14547) 806 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:38.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:37 smithi067 bash[17385]: cluster 2024-02-10T08:43:36.394054+0000 mgr.y (mgr.14547) 807 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:38.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:43:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:37] "GET /metrics HTTP/1.1" 200 192033 "" "Prometheus/2.33.4" 2024-02-10T08:43:38.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:37 smithi067 bash[24488]: cluster 2024-02-10T08:43:36.394054+0000 mgr.y (mgr.14547) 807 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:38.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:37 smithi134 bash[18722]: cluster 2024-02-10T08:43:36.394054+0000 mgr.y (mgr.14547) 807 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 169 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:40.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:39 smithi067 bash[17385]: cluster 2024-02-10T08:43:38.396079+0000 mgr.y (mgr.14547) 808 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:43:40.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:39 smithi067 bash[24488]: cluster 2024-02-10T08:43:38.396079+0000 mgr.y (mgr.14547) 808 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:43:40.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:39 smithi134 bash[18722]: cluster 2024-02-10T08:43:38.396079+0000 mgr.y (mgr.14547) 808 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:43:42.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:41 smithi067 bash[17385]: cluster 2024-02-10T08:43:40.397023+0000 mgr.y (mgr.14547) 809 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T08:43:42.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:41 smithi067 bash[24488]: cluster 2024-02-10T08:43:40.397023+0000 mgr.y (mgr.14547) 809 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T08:43:42.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:41 smithi134 bash[18722]: cluster 2024-02-10T08:43:40.397023+0000 mgr.y (mgr.14547) 809 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-10T08:43:43.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:43:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:43:43.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:42 smithi134 bash[18722]: cluster 2024-02-10T08:43:42.470068+0000 mon.a (mon.0) 3199 : cluster [WRN] pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' is full (reached quota's max_objects: 10) 2024-02-10T08:43:43.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:42 smithi134 bash[18722]: cluster 2024-02-10T08:43:42.470564+0000 mon.a (mon.0) 3200 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:43:43.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:42 smithi134 bash[18722]: cluster 2024-02-10T08:43:42.479613+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-10T08:43:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:42 smithi067 bash[17385]: cluster 2024-02-10T08:43:42.470068+0000 mon.a (mon.0) 3199 : cluster [WRN] pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' is full (reached quota's max_objects: 10) 2024-02-10T08:43:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:42 smithi067 bash[17385]: cluster 2024-02-10T08:43:42.470564+0000 mon.a (mon.0) 3200 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:43:43.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:42 smithi067 bash[17385]: cluster 2024-02-10T08:43:42.479613+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-10T08:43:43.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:42 smithi067 bash[24488]: cluster 2024-02-10T08:43:42.470068+0000 mon.a (mon.0) 3199 : cluster [WRN] pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' is full (reached quota's max_objects: 10) 2024-02-10T08:43:43.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:42 smithi067 bash[24488]: cluster 2024-02-10T08:43:42.470564+0000 mon.a (mon.0) 3200 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-10T08:43:43.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:42 smithi067 bash[24488]: cluster 2024-02-10T08:43:42.479613+0000 mon.a (mon.0) 3201 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-10T08:43:43.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:43:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:43:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:43 smithi067 bash[17385]: cluster 2024-02-10T08:43:42.399005+0000 mgr.y (mgr.14547) 810 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:43:44.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:43 smithi067 bash[17385]: audit 2024-02-10T08:43:42.650667+0000 mgr.y (mgr.14547) 811 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:43 smithi067 bash[24488]: cluster 2024-02-10T08:43:42.399005+0000 mgr.y (mgr.14547) 810 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:43:44.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:43 smithi067 bash[24488]: audit 2024-02-10T08:43:42.650667+0000 mgr.y (mgr.14547) 811 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:43 smithi134 bash[18722]: cluster 2024-02-10T08:43:42.399005+0000 mgr.y (mgr.14547) 810 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-10T08:43:44.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:43 smithi134 bash[18722]: audit 2024-02-10T08:43:42.650667+0000 mgr.y (mgr.14547) 811 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:45 smithi067 bash[17385]: cluster 2024-02-10T08:43:44.399773+0000 mgr.y (mgr.14547) 812 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:43:46.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:45 smithi067 bash[17385]: audit 2024-02-10T08:43:45.798414+0000 mon.a (mon.0) 3202 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:45 smithi067 bash[24488]: cluster 2024-02-10T08:43:44.399773+0000 mgr.y (mgr.14547) 812 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:43:46.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:45 smithi067 bash[24488]: audit 2024-02-10T08:43:45.798414+0000 mon.a (mon.0) 3202 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:46.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:45 smithi134 bash[18722]: cluster 2024-02-10T08:43:44.399773+0000 mgr.y (mgr.14547) 812 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:43:46.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:45 smithi134 bash[18722]: audit 2024-02-10T08:43:45.798414+0000 mon.a (mon.0) 3202 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:48.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:47 smithi067 bash[17385]: cluster 2024-02-10T08:43:46.400587+0000 mgr.y (mgr.14547) 813 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:43:48.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:43:47 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:47] "GET /metrics HTTP/1.1" 200 192061 "" "Prometheus/2.33.4" 2024-02-10T08:43:48.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:47 smithi067 bash[24488]: cluster 2024-02-10T08:43:46.400587+0000 mgr.y (mgr.14547) 813 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:43:48.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:47 smithi134 bash[18722]: cluster 2024-02-10T08:43:46.400587+0000 mgr.y (mgr.14547) 813 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-10T08:43:50.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:49 smithi067 bash[17385]: cluster 2024-02-10T08:43:48.402558+0000 mgr.y (mgr.14547) 814 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:50.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:49 smithi067 bash[24488]: cluster 2024-02-10T08:43:48.402558+0000 mgr.y (mgr.14547) 814 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:50.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:49 smithi134 bash[18722]: cluster 2024-02-10T08:43:48.402558+0000 mgr.y (mgr.14547) 814 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:52.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:51 smithi067 bash[17385]: cluster 2024-02-10T08:43:50.403379+0000 mgr.y (mgr.14547) 815 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:52.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:51 smithi067 bash[24488]: cluster 2024-02-10T08:43:50.403379+0000 mgr.y (mgr.14547) 815 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:52.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:51 smithi134 bash[18722]: cluster 2024-02-10T08:43:50.403379+0000 mgr.y (mgr.14547) 815 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:53.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:43:52 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:43:53.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:43:53 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:53] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:43:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:53 smithi067 bash[17385]: cluster 2024-02-10T08:43:52.405420+0000 mgr.y (mgr.14547) 816 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:54.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:53 smithi067 bash[17385]: audit 2024-02-10T08:43:52.653098+0000 mgr.y (mgr.14547) 817 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:54.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:53 smithi067 bash[24488]: cluster 2024-02-10T08:43:52.405420+0000 mgr.y (mgr.14547) 816 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:54.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:53 smithi067 bash[24488]: audit 2024-02-10T08:43:52.653098+0000 mgr.y (mgr.14547) 817 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:54.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:53 smithi134 bash[18722]: cluster 2024-02-10T08:43:52.405420+0000 mgr.y (mgr.14547) 816 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-10T08:43:54.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:53 smithi134 bash[18722]: audit 2024-02-10T08:43:52.653098+0000 mgr.y (mgr.14547) 817 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:43:56.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:55 smithi067 bash[17385]: cluster 2024-02-10T08:43:54.406202+0000 mgr.y (mgr.14547) 818 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-10T08:43:56.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:55 smithi067 bash[24488]: cluster 2024-02-10T08:43:54.406202+0000 mgr.y (mgr.14547) 818 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-10T08:43:56.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:55 smithi134 bash[18722]: cluster 2024-02-10T08:43:54.406202+0000 mgr.y (mgr.14547) 818 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-10T08:43:57.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:56 smithi067 bash[17385]: audit 2024-02-10T08:43:56.605089+0000 mon.c (mon.2) 1042 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:43:57.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:56 smithi067 bash[24488]: audit 2024-02-10T08:43:56.605089+0000 mon.c (mon.2) 1042 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:43:57.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:56 smithi134 bash[18722]: audit 2024-02-10T08:43:56.605089+0000 mon.c (mon.2) 1042 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T08:43:58.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:43:57] "GET /metrics HTTP/1.1" 200 192061 "" "Prometheus/2.33.4" 2024-02-10T08:43:58.123 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[24488]: cluster 2024-02-10T08:43:56.406997+0000 mgr.y (mgr.14547) 819 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[24488]: audit 2024-02-10T08:43:56.915854+0000 mon.c (mon.2) 1043 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[24488]: audit 2024-02-10T08:43:56.917105+0000 mon.c (mon.2) 1044 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[24488]: audit 2024-02-10T08:43:56.924630+0000 mon.a (mon.0) 3203 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[17385]: cluster 2024-02-10T08:43:56.406997+0000 mgr.y (mgr.14547) 819 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[17385]: audit 2024-02-10T08:43:56.915854+0000 mon.c (mon.2) 1043 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[17385]: audit 2024-02-10T08:43:56.917105+0000 mon.c (mon.2) 1044 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:43:58.124 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:57 smithi067 bash[17385]: audit 2024-02-10T08:43:56.924630+0000 mon.a (mon.0) 3203 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:43:58.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:57 smithi134 bash[18722]: cluster 2024-02-10T08:43:56.406997+0000 mgr.y (mgr.14547) 819 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:43:58.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:57 smithi134 bash[18722]: audit 2024-02-10T08:43:56.915854+0000 mon.c (mon.2) 1043 : audit [DBG] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T08:43:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:57 smithi134 bash[18722]: audit 2024-02-10T08:43:56.917105+0000 mon.c (mon.2) 1044 : audit [INF] from='mgr.14547 172.21.15.67:0/2414341934' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T08:43:58.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:57 smithi134 bash[18722]: audit 2024-02-10T08:43:56.924630+0000 mon.a (mon.0) 3203 : audit [INF] from='mgr.14547 ' entity='mgr.y' 2024-02-10T08:44:00.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:43:59 smithi067 bash[17385]: cluster 2024-02-10T08:43:58.408881+0000 mgr.y (mgr.14547) 820 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:00.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:43:59 smithi067 bash[24488]: cluster 2024-02-10T08:43:58.408881+0000 mgr.y (mgr.14547) 820 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:00.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:43:59 smithi134 bash[18722]: cluster 2024-02-10T08:43:58.408881+0000 mgr.y (mgr.14547) 820 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:02.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:01 smithi067 bash[17385]: cluster 2024-02-10T08:44:00.409702+0000 mgr.y (mgr.14547) 821 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:02.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:01 smithi067 bash[24488]: cluster 2024-02-10T08:44:00.409702+0000 mgr.y (mgr.14547) 821 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:02.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:01 smithi134 bash[18722]: cluster 2024-02-10T08:44:00.409702+0000 mgr.y (mgr.14547) 821 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:03.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:44:02 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:44:03.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:03 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:03] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:44:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:03 smithi067 bash[17385]: cluster 2024-02-10T08:44:02.411733+0000 mgr.y (mgr.14547) 822 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:04.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:03 smithi067 bash[17385]: audit 2024-02-10T08:44:02.661169+0000 mgr.y (mgr.14547) 823 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:03 smithi067 bash[24488]: cluster 2024-02-10T08:44:02.411733+0000 mgr.y (mgr.14547) 822 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:04.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:03 smithi067 bash[24488]: audit 2024-02-10T08:44:02.661169+0000 mgr.y (mgr.14547) 823 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:04.132 INFO:tasks.workunit.client.0.smithi067.stderr:+ rados -p 34c11e82-d78d-4916-9c14-acce32e59950 put threemore /etc/passwd 2024-02-10T08:44:04.178 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_bytes 0 2024-02-10T08:44:04.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:03 smithi134 bash[18722]: cluster 2024-02-10T08:44:02.411733+0000 mgr.y (mgr.14547) 822 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:04.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:03 smithi134 bash[18722]: audit 2024-02-10T08:44:02.661169+0000 mgr.y (mgr.14547) 823 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- 172.21.15.67:0/613007677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fc6c0 msgr2=0x7f52080f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- 172.21.15.67:0/613007677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fc6c0 0x7f52080f5f10 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 rx=0x7f51f8004870 tx=0x7f51f8009cb0).stop 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- 172.21.15.67:0/613007677 shutdown_connections 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- 172.21.15.67:0/613007677 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52080f65f0 0x7f5208102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- 172.21.15.67:0/613007677 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fc6c0 0x7f52080f5f10 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- 172.21.15.67:0/613007677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52080fbd60 0x7f52080fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- 172.21.15.67:0/613007677 >> 172.21.15.67:0/613007677 conn(0x7f520806db60 msgr2=0x7f520806df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- 172.21.15.67:0/613007677 shutdown_connections 2024-02-10T08:44:04.288 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- 172.21.15.67:0/613007677 wait complete. 2024-02-10T08:44:04.289 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 Processor -- start 2024-02-10T08:44:04.289 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- start start 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52080f65f0 0x7f520810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 0x7f520810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52080fc6c0 0x7f5208107bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52081080a0 con 0x7f52080fbd60 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5208108210 con 0x7f52080f65f0 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520efd4700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52081083d0 con 0x7f52080fc6c0 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f51fffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 0x7f520810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f51fffff700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 0x7f520810dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:33994/0 (socket says 172.21.15.67:33994) 2024-02-10T08:44:04.290 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f51fffff700 1 -- 172.21.15.67:0/2143997445 learned_addr learned my addr 172.21.15.67:0/2143997445 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:44:04.291 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520d571700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52080fc6c0 0x7f5208107bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:04.291 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.283+0000 7f520cd70700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52080f65f0 0x7f520810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:04.291 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fffff700 1 -- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52080fc6c0 msgr2=0x7f5208107bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fffff700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52080fc6c0 0x7f5208107bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fffff700 1 -- 172.21.15.67:0/2143997445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52080f65f0 msgr2=0x7f520810d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fffff700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52080f65f0 0x7f520810d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fffff700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f8005040 con 0x7f52080fbd60 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fffff700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 0x7f520810dee0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 rx=0x7f51f8000e50 tx=0x7f51f8009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800e030 con 0x7f52080fbd60 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5208108650 con 0x7f52080fbd60 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f51f80096d0 con 0x7f52080fbd60 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8018070 con 0x7f52080fbd60 2024-02-10T08:44:04.292 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5208114840 con 0x7f52080fbd60 2024-02-10T08:44:04.293 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52080606d0 con 0x7f52080fbd60 2024-02-10T08:44:04.293 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f51f8009240 con 0x7f52080fbd60 2024-02-10T08:44:04.294 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.287+0000 7f51fdffb700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f51f0061340 0x7f51f0063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:04.294 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.291+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(737..737 src has 1..737) v4 ==== 9171+0+0 (secure 0 0 0) 0x7f51f807e740 con 0x7f52080fbd60 2024-02-10T08:44:04.294 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.291+0000 7f520cd70700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f51f0061340 0x7f51f0063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:04.295 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.291+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=738}) v3 -- 0x7f51f006d0e0 con 0x7f52080fbd60 2024-02-10T08:44:04.295 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.291+0000 7f520cd70700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f51f0061340 0x7f51f0063800 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 rx=0x7f52040092d0 tx=0x7f520400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:04.303 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.299+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f51f804dd80 con 0x7f52080fbd60 2024-02-10T08:44:04.462 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.455+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f520805f020 con 0x7f52080fbd60 2024-02-10T08:44:04.872 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.867+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 7 ==== osd_map(738..738 src has 1..738) v4 ==== 620+0+0 (secure 0 0 0) 0x7f51f80094f0 con 0x7f52080fbd60 2024-02-10T08:44:04.872 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.867+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=739}) v3 -- 0x7f51f006de00 con 0x7f52080fbd60 2024-02-10T08:44:04.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.871+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v738) v1 ==== 217+0+0 (secure 0 0 0) 0x7f51f80513a0 con 0x7f52080fbd60 2024-02-10T08:44:04.978 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:04.971+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f520805f020 con 0x7f52080fbd60 2024-02-10T08:44:05.120 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:04 smithi067 bash[17385]: audit 2024-02-10T08:44:04.463038+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:44:05.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:04 smithi067 bash[24488]: audit 2024-02-10T08:44:04.463038+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:44:05.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:04 smithi134 bash[18722]: audit 2024-02-10T08:44:04.463038+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:44:05.875 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.871+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 9 ==== osd_map(739..739 src has 1..739) v4 ==== 620+0+0 (secure 0 0 0) 0x7f51f80519e0 con 0x7f52080fbd60 2024-02-10T08:44:05.876 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.871+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=740}) v3 -- 0x7f51f006e2d0 con 0x7f52080fbd60 2024-02-10T08:44:05.882 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.879+0000 7f51fdffb700 1 -- 172.21.15.67:0/2143997445 <== mon.0 v2:172.21.15.67:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v739) v1 ==== 217+0+0 (secure 0 0 0) 0x7f51f8051580 con 0x7f52080fbd60 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.879+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f51f0061340 msgr2=0x7f51f0063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.879+0000 7f520efd4700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f51f0061340 0x7f51f0063800 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 rx=0x7f52040092d0 tx=0x7f520400b040).stop 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.879+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 msgr2=0x7f520810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.879+0000 7f520efd4700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 0x7f520810dee0 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 rx=0x7f51f8000e50 tx=0x7f51f8009cb0).stop 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 shutdown_connections 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f51f0061340 0x7f51f0063800 unknown :-1 s=CLOSED pgs=3594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f52080fc6c0 0x7f5208107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f52080fbd60 0x7f520810dee0 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 --2- 172.21.15.67:0/2143997445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52080f65f0 0x7f520810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:05.887 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 >> 172.21.15.67:0/2143997445 conn(0x7f520806db60 msgr2=0x7f52080f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:05.888 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 shutdown_connections 2024-02-10T08:44:05.888 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.883+0000 7f520efd4700 1 -- 172.21.15.67:0/2143997445 wait complete. 2024-02-10T08:44:05.889 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_bytes = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:44:05.902 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool set-quota 34c11e82-d78d-4916-9c14-acce32e59950 max_objects 0 2024-02-10T08:44:06.004 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1574166698 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200fd8f0 msgr2=0x7fbb200fdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:06.004 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1574166698 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200fd8f0 0x7fbb200fdcd0 secure :-1 s=READY pgs=3075 cs=0 l=1 rev1=1 rx=0x7fbb10009be0 tx=0x7fbb10004d10).stop 2024-02-10T08:44:06.004 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1574166698 shutdown_connections 2024-02-10T08:44:06.004 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1574166698 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200f81c0 0x7fbb20104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:06.004 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1574166698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb200fe250 0x7fbb200f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:06.005 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1574166698 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200fd8f0 0x7fbb200fdcd0 unknown :-1 s=CLOSED pgs=3075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:06.005 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1574166698 >> 172.21.15.67:0/1574166698 conn(0x7fbb2006f900 msgr2=0x7fbb2006fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:06.005 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1574166698 shutdown_connections 2024-02-10T08:44:06.005 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1574166698 wait complete. 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 Processor -- start 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- start start 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 0x7fbb2010dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200fd8f0 0x7fbb2010e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb200fe250 0x7fbb20107ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20108390 con 0x7fbb200fd8f0 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb20108500 con 0x7fbb200fe250 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb25d0c700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201086c0 con 0x7fbb200f81c0 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200fd8f0 0x7fbb2010e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 0x7fbb2010dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:06.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 0x7fbb2010dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:42054/0 (socket says 172.21.15.67:42054) 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 -- 172.21.15.67:0/1834298964 learned_addr learned my addr 172.21.15.67:0/1834298964 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1ffff700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb200fe250 0x7fbb20107ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1effd700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200fd8f0 0x7fbb2010e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:34016/0 (socket says 172.21.15.67:34016) 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 -- 172.21.15.67:0/1834298964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb200fe250 msgr2=0x7fbb20107ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb200fe250 0x7fbb20107ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 -- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200fd8f0 msgr2=0x7fbb2010e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200fd8f0 0x7fbb2010e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:05.999+0000 7fbb1f7fe700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18005040 con 0x7fbb200f81c0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1f7fe700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 0x7fbb2010dce0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 rx=0x7fbb10004f10 tx=0x7fbb10004d10).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000e030 con 0x7fbb200f81c0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb100097c0 con 0x7fbb200f81c0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7fbb10007da0 con 0x7fbb200f81c0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb20108bb0 con 0x7fbb200f81c0 2024-02-10T08:44:06.008 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10018070 con 0x7fbb200f81c0 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7fbb10007910 con 0x7fbb200f81c0 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb2005f510 con 0x7fbb200f81c0 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fbb08061340 0x7fbb08063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(739..739 src has 1..739) v4 ==== 9171+0+0 (secure 0 0 0) 0x7fbb1007ec00 con 0x7fbb200f81c0 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1effd700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fbb08061340 0x7fbb08063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=740}) v3 -- 0x7fbb0806d0e0 con 0x7fbb200f81c0 2024-02-10T08:44:06.010 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.003+0000 7fbb1effd700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fbb08061340 0x7fbb08063800 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 rx=0x7fbb20109640 tx=0x7fbb1400b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:06.015 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.011+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7fbb1004e1c0 con 0x7fbb200f81c0 2024-02-10T08:44:06.171 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.167+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fbb20106c50 con 0x7fbb200f81c0 2024-02-10T08:44:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:05 smithi134 bash[18722]: cluster 2024-02-10T08:44:04.412533+0000 mgr.y (mgr.14547) 824 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:05 smithi134 bash[18722]: audit 2024-02-10T08:44:04.876194+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:44:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:05 smithi134 bash[18722]: cluster 2024-02-10T08:44:04.876347+0000 mon.a (mon.0) 3206 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-10T08:44:06.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:05 smithi134 bash[18722]: audit 2024-02-10T08:44:04.978640+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:44:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[17385]: cluster 2024-02-10T08:44:04.412533+0000 mgr.y (mgr.14547) 824 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[17385]: audit 2024-02-10T08:44:04.876194+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:44:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[17385]: cluster 2024-02-10T08:44:04.876347+0000 mon.a (mon.0) 3206 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-10T08:44:06.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[17385]: audit 2024-02-10T08:44:04.978640+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:44:06.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[24488]: cluster 2024-02-10T08:44:04.412533+0000 mgr.y (mgr.14547) 824 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[24488]: audit 2024-02-10T08:44:04.876194+0000 mon.a (mon.0) 3205 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:44:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[24488]: cluster 2024-02-10T08:44:04.876347+0000 mon.a (mon.0) 3206 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-10T08:44:06.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:05 smithi067 bash[24488]: audit 2024-02-10T08:44:04.978640+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-10T08:44:06.901 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.895+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 7 ==== osd_map(740..740 src has 1..740) v4 ==== 620+0+0 (secure 0 0 0) 0x7fbb10051e20 con 0x7fbb200f81c0 2024-02-10T08:44:06.902 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.895+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=741}) v3 -- 0x7fbb0806de00 con 0x7fbb200f81c0 2024-02-10T08:44:06.907 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:06.903+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v740) v1 ==== 221+0+0 (secure 0 0 0) 0x7fbb100517e0 con 0x7fbb200f81c0 2024-02-10T08:44:07.007 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.003+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fbb20106c20 con 0x7fbb200f81c0 2024-02-10T08:44:07.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:06 smithi134 bash[18722]: audit 2024-02-10T08:44:05.882442+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:44:07.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:06 smithi134 bash[18722]: cluster 2024-02-10T08:44:05.882841+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-10T08:44:07.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:06 smithi134 bash[18722]: audit 2024-02-10T08:44:06.171962+0000 mon.c (mon.2) 1045 : audit [INF] from='client.? 172.21.15.67:0/1834298964' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:07.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:06 smithi134 bash[18722]: audit 2024-02-10T08:44:06.172977+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:07.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[17385]: audit 2024-02-10T08:44:05.882442+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:44:07.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[17385]: cluster 2024-02-10T08:44:05.882841+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-10T08:44:07.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[17385]: audit 2024-02-10T08:44:06.171962+0000 mon.c (mon.2) 1045 : audit [INF] from='client.? 172.21.15.67:0/1834298964' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:07.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[17385]: audit 2024-02-10T08:44:06.172977+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[24488]: audit 2024-02-10T08:44:05.882442+0000 mon.a (mon.0) 3208 : audit [INF] from='client.? 172.21.15.67:0/2143997445' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_bytes", "val": "0"}]': finished 2024-02-10T08:44:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[24488]: cluster 2024-02-10T08:44:05.882841+0000 mon.a (mon.0) 3209 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-10T08:44:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[24488]: audit 2024-02-10T08:44:06.171962+0000 mon.c (mon.2) 1045 : audit [INF] from='client.? 172.21.15.67:0/1834298964' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:07.371 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:06 smithi067 bash[24488]: audit 2024-02-10T08:44:06.172977+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:07.918 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.911+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 9 ==== osd_map(741..741 src has 1..741) v4 ==== 620+0+0 (secure 0 0 0) 0x7fbb100517e0 con 0x7fbb200f81c0 2024-02-10T08:44:07.918 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.911+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=742}) v3 -- 0x7fbb0806e2d0 con 0x7fbb200f81c0 2024-02-10T08:44:07.924 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.919+0000 7fbb1cff9700 1 -- 172.21.15.67:0/1834298964 <== mon.2 v2:172.21.15.67:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 v741) v1 ==== 221+0+0 (secure 0 0 0) 0x7fbb10034060 con 0x7fbb200f81c0 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fbb08061340 msgr2=0x7fbb08063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fbb08061340 0x7fbb08063800 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 rx=0x7fbb20109640 tx=0x7fbb1400b040).stop 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 msgr2=0x7fbb2010dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 0x7fbb2010dce0 secure :-1 s=READY pgs=3076 cs=0 l=1 rev1=1 rx=0x7fbb10004f10 tx=0x7fbb10004d10).stop 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 shutdown_connections 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7fbb08061340 0x7fbb08063800 unknown :-1 s=CLOSED pgs=3595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb200fe250 0x7fbb20107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:07.927 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7fbb200fd8f0 0x7fbb2010e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:07.928 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 --2- 172.21.15.67:0/1834298964 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7fbb200f81c0 0x7fbb2010dce0 unknown :-1 s=CLOSED pgs=3076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:07.928 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 >> 172.21.15.67:0/1834298964 conn(0x7fbb2006f900 msgr2=0x7fbb200f7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:07.928 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 shutdown_connections 2024-02-10T08:44:07.928 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:07.923+0000 7fbb25d0c700 1 -- 172.21.15.67:0/1834298964 wait complete. 2024-02-10T08:44:07.929 INFO:tasks.workunit.client.0.smithi067.stderr:set-quota max_objects = 0 for pool 34c11e82-d78d-4916-9c14-acce32e59950 2024-02-10T08:44:07.939 INFO:tasks.workunit.client.0.smithi067.stderr:+ sleep 30 2024-02-10T08:44:08.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:07] "GET /metrics HTTP/1.1" 200 192063 "" "Prometheus/2.33.4" 2024-02-10T08:44:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[24488]: cluster 2024-02-10T08:44:06.413387+0000 mgr.y (mgr.14547) 825 : cluster [DBG] pgmap v1315: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:44:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[24488]: audit 2024-02-10T08:44:06.903608+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:44:08.120 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[24488]: cluster 2024-02-10T08:44:06.903749+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[24488]: audit 2024-02-10T08:44:07.008423+0000 mon.c (mon.2) 1046 : audit [INF] from='client.? 172.21.15.67:0/1834298964' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[24488]: audit 2024-02-10T08:44:07.009165+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[17385]: cluster 2024-02-10T08:44:06.413387+0000 mgr.y (mgr.14547) 825 : cluster [DBG] pgmap v1315: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[17385]: audit 2024-02-10T08:44:06.903608+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[17385]: cluster 2024-02-10T08:44:06.903749+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[17385]: audit 2024-02-10T08:44:07.008423+0000 mon.c (mon.2) 1046 : audit [INF] from='client.? 172.21.15.67:0/1834298964' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:08.121 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:07 smithi067 bash[17385]: audit 2024-02-10T08:44:07.009165+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:08.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:07 smithi134 bash[18722]: cluster 2024-02-10T08:44:06.413387+0000 mgr.y (mgr.14547) 825 : cluster [DBG] pgmap v1315: 188 pgs: 188 active+clean; 286 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-10T08:44:08.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:07 smithi134 bash[18722]: audit 2024-02-10T08:44:06.903608+0000 mon.a (mon.0) 3211 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:44:08.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:07 smithi134 bash[18722]: cluster 2024-02-10T08:44:06.903749+0000 mon.a (mon.0) 3212 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-10T08:44:08.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:07 smithi134 bash[18722]: audit 2024-02-10T08:44:07.008423+0000 mon.c (mon.2) 1046 : audit [INF] from='client.? 172.21.15.67:0/1834298964' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:08.277 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:07 smithi134 bash[18722]: audit 2024-02-10T08:44:07.009165+0000 mon.a (mon.0) 3213 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]: dispatch 2024-02-10T08:44:09.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:08 smithi134 bash[18722]: audit 2024-02-10T08:44:07.921646+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:44:09.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:08 smithi134 bash[18722]: cluster 2024-02-10T08:44:07.921760+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-10T08:44:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:08 smithi067 bash[17385]: audit 2024-02-10T08:44:07.921646+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:44:09.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:08 smithi067 bash[17385]: cluster 2024-02-10T08:44:07.921760+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-10T08:44:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:08 smithi067 bash[24488]: audit 2024-02-10T08:44:07.921646+0000 mon.a (mon.0) 3214 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "field": "max_objects", "val": "0"}]': finished 2024-02-10T08:44:09.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:08 smithi067 bash[24488]: cluster 2024-02-10T08:44:07.921760+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-10T08:44:10.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:09 smithi134 bash[18722]: cluster 2024-02-10T08:44:08.414348+0000 mgr.y (mgr.14547) 826 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:44:10.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:09 smithi067 bash[17385]: cluster 2024-02-10T08:44:08.414348+0000 mgr.y (mgr.14547) 826 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:44:10.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:09 smithi067 bash[24488]: cluster 2024-02-10T08:44:08.414348+0000 mgr.y (mgr.14547) 826 : cluster [DBG] pgmap v1318: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-10T08:44:12.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:11 smithi134 bash[18722]: cluster 2024-02-10T08:44:10.415267+0000 mgr.y (mgr.14547) 827 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 923 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-02-10T08:44:12.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:11 smithi067 bash[17385]: cluster 2024-02-10T08:44:10.415267+0000 mgr.y (mgr.14547) 827 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 923 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-02-10T08:44:12.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:11 smithi067 bash[24488]: cluster 2024-02-10T08:44:10.415267+0000 mgr.y (mgr.14547) 827 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 923 B/s rd, 2.2 KiB/s wr, 1 op/s 2024-02-10T08:44:13.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:44:12 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:44:13.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:13 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:13] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:44:14.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:13 smithi134 bash[18722]: cluster 2024-02-10T08:44:12.417237+0000 mgr.y (mgr.14547) 828 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2024-02-10T08:44:14.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:13 smithi134 bash[18722]: audit 2024-02-10T08:44:12.670919+0000 mgr.y (mgr.14547) 829 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:13 smithi067 bash[17385]: cluster 2024-02-10T08:44:12.417237+0000 mgr.y (mgr.14547) 828 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2024-02-10T08:44:14.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:13 smithi067 bash[17385]: audit 2024-02-10T08:44:12.670919+0000 mgr.y (mgr.14547) 829 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:13 smithi067 bash[24488]: cluster 2024-02-10T08:44:12.417237+0000 mgr.y (mgr.14547) 828 : cluster [DBG] pgmap v1320: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.8 KiB/s wr, 1 op/s 2024-02-10T08:44:14.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:13 smithi067 bash[24488]: audit 2024-02-10T08:44:12.670919+0000 mgr.y (mgr.14547) 829 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:15.276 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:14 smithi134 bash[18722]: cluster 2024-02-10T08:44:14.418168+0000 mgr.y (mgr.14547) 830 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T08:44:15.370 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:14 smithi067 bash[24488]: cluster 2024-02-10T08:44:14.418168+0000 mgr.y (mgr.14547) 830 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T08:44:15.370 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:14 smithi067 bash[17385]: cluster 2024-02-10T08:44:14.418168+0000 mgr.y (mgr.14547) 830 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-10T08:44:17.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:17 smithi134 bash[18722]: cluster 2024-02-10T08:44:16.419148+0000 mgr.y (mgr.14547) 831 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 538 B/s rd, 0 op/s 2024-02-10T08:44:17.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:17 smithi067 bash[17385]: cluster 2024-02-10T08:44:16.419148+0000 mgr.y (mgr.14547) 831 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 538 B/s rd, 0 op/s 2024-02-10T08:44:17.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:17 smithi067 bash[24488]: cluster 2024-02-10T08:44:16.419148+0000 mgr.y (mgr.14547) 831 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 538 B/s rd, 0 op/s 2024-02-10T08:44:18.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:17 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:17] "GET /metrics HTTP/1.1" 200 192073 "" "Prometheus/2.33.4" 2024-02-10T08:44:19.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:19 smithi134 bash[18722]: cluster 2024-02-10T08:44:18.421293+0000 mgr.y (mgr.14547) 832 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 975 B/s rd, 0 op/s 2024-02-10T08:44:19.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:19 smithi067 bash[24488]: cluster 2024-02-10T08:44:18.421293+0000 mgr.y (mgr.14547) 832 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 975 B/s rd, 0 op/s 2024-02-10T08:44:19.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:19 smithi067 bash[17385]: cluster 2024-02-10T08:44:18.421293+0000 mgr.y (mgr.14547) 832 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 975 B/s rd, 0 op/s 2024-02-10T08:44:21.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:21 smithi134 bash[18722]: cluster 2024-02-10T08:44:20.422168+0000 mgr.y (mgr.14547) 833 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:21.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:21 smithi067 bash[17385]: cluster 2024-02-10T08:44:20.422168+0000 mgr.y (mgr.14547) 833 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:21.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:21 smithi067 bash[24488]: cluster 2024-02-10T08:44:20.422168+0000 mgr.y (mgr.14547) 833 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:23.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:44:22 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:44:23.501 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:23 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:23] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:44:23.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:23 smithi134 bash[18722]: cluster 2024-02-10T08:44:22.424176+0000 mgr.y (mgr.14547) 834 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:23.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:23 smithi134 bash[18722]: audit 2024-02-10T08:44:22.675592+0000 mgr.y (mgr.14547) 835 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:23 smithi067 bash[17385]: cluster 2024-02-10T08:44:22.424176+0000 mgr.y (mgr.14547) 834 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:23.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:23 smithi067 bash[17385]: audit 2024-02-10T08:44:22.675592+0000 mgr.y (mgr.14547) 835 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:23.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:23 smithi067 bash[24488]: cluster 2024-02-10T08:44:22.424176+0000 mgr.y (mgr.14547) 834 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:23.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:23 smithi067 bash[24488]: audit 2024-02-10T08:44:22.675592+0000 mgr.y (mgr.14547) 835 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:25.779 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:25 smithi134 bash[18722]: cluster 2024-02-10T08:44:24.424972+0000 mgr.y (mgr.14547) 836 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:25.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:25 smithi067 bash[17385]: cluster 2024-02-10T08:44:24.424972+0000 mgr.y (mgr.14547) 836 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:25.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:25 smithi067 bash[24488]: cluster 2024-02-10T08:44:24.424972+0000 mgr.y (mgr.14547) 836 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:27.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:27 smithi134 bash[18722]: cluster 2024-02-10T08:44:26.425774+0000 mgr.y (mgr.14547) 837 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:27.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:27 smithi067 bash[17385]: cluster 2024-02-10T08:44:26.425774+0000 mgr.y (mgr.14547) 837 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:27.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:27 smithi067 bash[24488]: cluster 2024-02-10T08:44:26.425774+0000 mgr.y (mgr.14547) 837 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:28.120 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:27 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:27] "GET /metrics HTTP/1.1" 200 192073 "" "Prometheus/2.33.4" 2024-02-10T08:44:29.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:29 smithi134 bash[18722]: cluster 2024-02-10T08:44:28.427897+0000 mgr.y (mgr.14547) 838 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:29.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:29 smithi067 bash[17385]: cluster 2024-02-10T08:44:28.427897+0000 mgr.y (mgr.14547) 838 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:29.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:29 smithi067 bash[24488]: cluster 2024-02-10T08:44:28.427897+0000 mgr.y (mgr.14547) 838 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:31.776 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:31 smithi134 bash[18722]: cluster 2024-02-10T08:44:30.428708+0000 mgr.y (mgr.14547) 839 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:31.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:31 smithi067 bash[17385]: cluster 2024-02-10T08:44:30.428708+0000 mgr.y (mgr.14547) 839 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:31.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:31 smithi067 bash[24488]: cluster 2024-02-10T08:44:30.428708+0000 mgr.y (mgr.14547) 839 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:33.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:44:32 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:44:33.526 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:33 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:33] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:44:33.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:33 smithi067 bash[24488]: cluster 2024-02-10T08:44:32.430955+0000 mgr.y (mgr.14547) 840 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:33.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:33 smithi067 bash[24488]: audit 2024-02-10T08:44:32.679092+0000 mgr.y (mgr.14547) 841 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:33 smithi067 bash[17385]: cluster 2024-02-10T08:44:32.430955+0000 mgr.y (mgr.14547) 840 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:33.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:33 smithi067 bash[17385]: audit 2024-02-10T08:44:32.679092+0000 mgr.y (mgr.14547) 841 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:34.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:33 smithi134 bash[18722]: cluster 2024-02-10T08:44:32.430955+0000 mgr.y (mgr.14547) 840 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:34.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:33 smithi134 bash[18722]: audit 2024-02-10T08:44:32.679092+0000 mgr.y (mgr.14547) 841 : audit [DBG] from='client.24439 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-10T08:44:35.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:35 smithi067 bash[17385]: cluster 2024-02-10T08:44:34.431731+0000 mgr.y (mgr.14547) 842 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:35.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:35 smithi067 bash[24488]: cluster 2024-02-10T08:44:34.431731+0000 mgr.y (mgr.14547) 842 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:36.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:35 smithi134 bash[18722]: cluster 2024-02-10T08:44:34.431731+0000 mgr.y (mgr.14547) 842 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:37.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:37 smithi067 bash[17385]: cluster 2024-02-10T08:44:36.432600+0000 mgr.y (mgr.14547) 843 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:37.860 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:37 smithi067 bash[24488]: cluster 2024-02-10T08:44:36.432600+0000 mgr.y (mgr.14547) 843 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:37.940 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool delete 34c11e82-d78d-4916-9c14-acce32e59950 34c11e82-d78d-4916-9c14-acce32e59950 --yes-i-really-really-mean-it 2024-02-10T08:44:38.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:37 smithi134 bash[18722]: cluster 2024-02-10T08:44:36.432600+0000 mgr.y (mgr.14547) 843 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-10T08:44:38.056 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/528302296 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 msgr2=0x7f8aa00f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/528302296 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6680 secure :-1 s=READY pgs=3077 cs=0 l=1 rev1=1 rx=0x7f8a98004870 tx=0x7f8a98009cb0).stop 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/528302296 shutdown_connections 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/528302296 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6680 unknown :-1 s=CLOSED pgs=3077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/528302296 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/528302296 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 0x7f8aa00fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/528302296 >> 172.21.15.67:0/528302296 conn(0x7f8aa006fc20 msgr2=0x7f8aa00f3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/528302296 shutdown_connections 2024-02-10T08:44:38.057 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/528302296 wait complete. 2024-02-10T08:44:38.058 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 Processor -- start 2024-02-10T08:44:38.058 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- start start 2024-02-10T08:44:38.058 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.058 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 0x7f8aa00f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.058 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0115d70 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0115ee0 con 0x7f8aa00fd6e0 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.050+0000 7f8aa4b2e700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01160a0 con 0x7f8aa00f6220 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3300/0 says I am v2:172.21.15.67:45302/0 (socket says 172.21.15.67:45302) 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 -- 172.21.15.67:0/2791051801 learned_addr learned my addr 172.21.15.67:0/2791051801 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:44:38.059 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9e59c700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.060 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9dd9b700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 0x7f8aa00f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.060 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 -- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 msgr2=0x7f8aa00f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 -- 172.21.15.67:0/2791051801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 msgr2=0x7f8aa00f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 0x7f8aa00f7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98005040 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9dd9b700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 0x7f8aa00f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9e59c700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a9ed9d700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f7900 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7f8a98005010 tx=0x7f8a98009cb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800e030 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f8a9801d420 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa0116320 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98018070 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.061 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa0116740 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.062 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.054+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa005fc90 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.062 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.058+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f8a98009450 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.063 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.058+0000 7f8a8f7fe700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a88061340 0x7f8a88063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.063 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.058+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 5 ==== osd_map(741..741 src has 1..741) v4 ==== 9171+0+0 (secure 0 0 0) 0x7f8a9807ea20 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.063 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.058+0000 7f8a9e59c700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a88061340 0x7f8a88063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.063 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.058+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=742}) v3 -- 0x7f8a8806d0e0 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.064 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.058+0000 7f8a9e59c700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a88061340 0x7f8a88063800 secure :-1 s=READY pgs=3596 cs=0 l=1 rev1=1 rx=0x7f8aa0060670 tx=0x7f8a90004d00).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:38.068 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.062+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f8a9804e060 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.121 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:37 smithi067 bash[17647]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:37] "GET /metrics HTTP/1.1" 200 192069 "" "Prometheus/2.33.4" 2024-02-10T08:44:38.310 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.222+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8aa00f8400 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.554 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.546+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 7 ==== osd_map(742..742 src has 1..742) v4 ==== 288+0+0 (secure 0 0 0) 0x7f8a98009700 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.554 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.546+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_subscribe({osdmap=743}) v3 -- 0x7f8a8806db20 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.556 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.550+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]=0 pool '34c11e82-d78d-4916-9c14-acce32e59950' removed v742) v1 ==== 248+0+0 (secure 0 0 0) 0x7f8a98051680 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.659 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.654+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8aa00f8400 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.660 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.654+0000 7f8a8f7fe700 1 -- 172.21.15.67:0/2791051801 <== mon.0 v2:172.21.15.67:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]=0 pool '34c11e82-d78d-4916-9c14-acce32e59950' does not exist v742) v1 ==== 255+0+0 (secure 0 0 0) 0x7f8a98051860 con 0x7f8aa00fdfb0 2024-02-10T08:44:38.664 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a88061340 msgr2=0x7f8a88063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.664 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a88061340 0x7f8a88063800 secure :-1 s=READY pgs=3596 cs=0 l=1 rev1=1 rx=0x7f8aa0060670 tx=0x7f8a90004d00).stop 2024-02-10T08:44:38.664 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 msgr2=0x7f8aa00f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f7900 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 rx=0x7f8a98005010 tx=0x7f8a98009cb0).stop 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 shutdown_connections 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f8a88061340 0x7f8a88063800 unknown :-1 s=CLOSED pgs=3596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f8aa00fdfb0 0x7f8aa00f7900 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8aa00fd6e0 0x7f8aa00f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 --2- 172.21.15.67:0/2791051801 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f8aa00f6220 0x7f8aa00f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 >> 172.21.15.67:0/2791051801 conn(0x7f8aa006fc20 msgr2=0x7f8aa006cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 shutdown_connections 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.658+0000 7f8aa4b2e700 1 -- 172.21.15.67:0/2791051801 wait complete. 2024-02-10T08:44:38.665 INFO:tasks.workunit.client.0.smithi067.stderr:pool '34c11e82-d78d-4916-9c14-acce32e59950' does not exist 2024-02-10T08:44:38.678 INFO:tasks.workunit.client.0.smithi067.stderr:+ ceph osd pool delete 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 1b373566-6b27-4e28-8afd-e7e8ebc4bc37 --yes-i-really-really-mean-it 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/92457460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fc630 msgr2=0x7f78300fca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- 172.21.15.67:0/92457460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fc630 0x7f78300fca10 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 rx=0x7f78200047e0 tx=0x7f7820009cb0).stop 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/92457460 shutdown_connections 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- 172.21.15.67:0/92457460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300f7420 0x7f7830103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- 172.21.15.67:0/92457460 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300fcf90 0x7f78300f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- 172.21.15.67:0/92457460 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fc630 0x7f78300fca10 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/92457460 >> 172.21.15.67:0/92457460 conn(0x7f783006ca40 msgr2=0x7f78300f3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/92457460 shutdown_connections 2024-02-10T08:44:38.779 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/92457460 wait complete. 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 Processor -- start 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- start start 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 0x7f783010fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300fc630 0x7f7830110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 --2- >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fcf90 0x7f7830109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- --> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783010a420 con 0x7f78300fcf90 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783010a590 con 0x7f78300fc630 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783010a750 con 0x7f78300f7420 2024-02-10T08:44:38.782 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 0x7f783010fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 --2- >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 0x7f783010fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.67:3301/0 says I am v2:172.21.15.67:46094/0 (socket says 172.21.15.67:46094) 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 -- 172.21.15.67:0/2736237525 learned_addr learned my addr 172.21.15.67:0/2736237525 (peer_addr_for_me v2:172.21.15.67:0/0) 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f78349e6700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fcf90 0x7f7830109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782f7fe700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300fc630 0x7f7830110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 -- 172.21.15.67:0/2736237525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300fc630 msgr2=0x7f7830110260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300fc630 0x7f7830110260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 -- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fcf90 msgr2=0x7f7830109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fcf90 0x7f7830109f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7820005040 con 0x7f78300f7420 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782f7fe700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300fc630 0x7f7830110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782ffff700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 0x7f783010fd70 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7f78200095b0 tx=0x7f782000b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820019070 con 0x7f78300f7420 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783010a9d0 con 0x7f78300f7420 2024-02-10T08:44:38.783 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 2 ==== config(23 keys) v1 ==== 1041+0+0 (secure 0 0 0) 0x7f7820013380 con 0x7f78300f7420 2024-02-10T08:44:38.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782001eaf0 con 0x7f78300f7420 2024-02-10T08:44:38.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.774+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7830116bc0 con 0x7f78300f7420 2024-02-10T08:44:38.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78300470f0 con 0x7f78300f7420 2024-02-10T08:44:38.784 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 4 ==== mgrmap(e 20) v1 ==== 77121+0+0 (secure 0 0 0) 0x7f7820013940 con 0x7f78300f7420 2024-02-10T08:44:38.785 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f782d7fa700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f781c061340 0x7f781c063800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2024-02-10T08:44:38.785 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 5 ==== osd_map(742..742 src has 1..742) v4 ==== 8783+0+0 (secure 0 0 0) 0x7f782007e430 con 0x7f78300f7420 2024-02-10T08:44:38.785 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f782f7fe700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f781c061340 0x7f781c063800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2024-02-10T08:44:38.785 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=743}) v3 -- 0x7f781c06c8c0 con 0x7f78300f7420 2024-02-10T08:44:38.786 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.778+0000 7f782f7fe700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f781c061340 0x7f781c063800 secure :-1 s=READY pgs=3597 cs=0 l=1 rev1=1 rx=0x7f78180043d0 tx=0x7f781800b040).ready entity=mgr.14547 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-10T08:44:38.792 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.786+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (secure 0 0 0) 0x7f782004dc80 con 0x7f78300f7420 2024-02-10T08:44:38.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:38 smithi067 bash[17385]: audit 2024-02-10T08:44:38.227778+0000 mon.a (mon.0) 3216 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:38.870 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:38 smithi067 bash[24488]: audit 2024-02-10T08:44:38.227778+0000 mon.a (mon.0) 3216 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:38.948 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:38.942+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f783005fc90 con 0x7f78300f7420 2024-02-10T08:44:39.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:38 smithi134 bash[18722]: audit 2024-02-10T08:44:38.227778+0000 mon.a (mon.0) 3216 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:39.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.558+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 7 ==== osd_map(743..743 src has 1..743) v4 ==== 288+0+0 (secure 0 0 0) 0x7f782001e420 con 0x7f78300f7420 2024-02-10T08:44:39.565 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.558+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_subscribe({osdmap=744}) v3 -- 0x7f781c06d2c0 con 0x7f78300f7420 2024-02-10T08:44:39.571 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.566+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]=0 pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' removed v743) v1 ==== 248+0+0 (secure 0 0 0) 0x7f78200043d0 con 0x7f78300f7420 2024-02-10T08:44:39.671 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.666+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 --> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f783010b410 con 0x7f78300f7420 2024-02-10T08:44:39.673 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.666+0000 7f782d7fa700 1 -- 172.21.15.67:0/2736237525 <== mon.2 v2:172.21.15.67:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]=0 pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' does not exist v743) v1 ==== 255+0+0 (secure 0 0 0) 0x7f7820013bf0 con 0x7f78300f7420 2024-02-10T08:44:39.675 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f781c061340 msgr2=0x7f781c063800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:39.676 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f781c061340 0x7f781c063800 secure :-1 s=READY pgs=3597 cs=0 l=1 rev1=1 rx=0x7f78180043d0 tx=0x7f781800b040).stop 2024-02-10T08:44:39.676 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 msgr2=0x7f783010fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-10T08:44:39.676 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 0x7f783010fd70 secure :-1 s=READY pgs=3078 cs=0 l=1 rev1=1 rx=0x7f78200095b0 tx=0x7f782000b040).stop 2024-02-10T08:44:39.676 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 shutdown_connections 2024-02-10T08:44:39.676 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:6800/4090043441,v1:172.21.15.67:6801/4090043441] conn(0x7f781c061340 0x7f781c063800 unknown :-1 s=CLOSED pgs=3597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] conn(0x7f78300fcf90 0x7f7830109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78300fc630 0x7f7830110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 --2- 172.21.15.67:0/2736237525 >> [v2:172.21.15.67:3301/0,v1:172.21.15.67:6790/0] conn(0x7f78300f7420 0x7f783010fd70 unknown :-1 s=CLOSED pgs=3078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 >> 172.21.15.67:0/2736237525 conn(0x7f783006ca40 msgr2=0x7f78300f5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 shutdown_connections 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:2024-02-10T08:44:39.670+0000 7f7836449700 1 -- 172.21.15.67:0/2736237525 wait complete. 2024-02-10T08:44:39.677 INFO:tasks.workunit.client.0.smithi067.stderr:pool '1b373566-6b27-4e28-8afd-e7e8ebc4bc37' does not exist 2024-02-10T08:44:39.688 INFO:tasks.workunit.client.0.smithi067.stdout:OK 2024-02-10T08:44:39.689 INFO:tasks.workunit.client.0.smithi067.stderr:+ echo OK 2024-02-10T08:44:39.689 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-10T08:44:39.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-10T08:44:39.708 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-10T08:44:39.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-10T08:44:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[17385]: cluster 2024-02-10T08:44:38.434669+0000 mgr.y (mgr.14547) 844 : cluster [DBG] pgmap v1333: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[17385]: audit 2024-02-10T08:44:38.556859+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:44:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[17385]: cluster 2024-02-10T08:44:38.557060+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-10T08:44:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[17385]: audit 2024-02-10T08:44:38.660471+0000 mon.a (mon.0) 3219 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:39.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[17385]: audit 2024-02-10T08:44:38.949494+0000 mon.c (mon.2) 1047 : audit [INF] from='client.? 172.21.15.67:0/2736237525' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[17385]: audit 2024-02-10T08:44:38.950133+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[24488]: cluster 2024-02-10T08:44:38.434669+0000 mgr.y (mgr.14547) 844 : cluster [DBG] pgmap v1333: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[24488]: audit 2024-02-10T08:44:38.556859+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[24488]: cluster 2024-02-10T08:44:38.557060+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[24488]: audit 2024-02-10T08:44:38.660471+0000 mon.a (mon.0) 3219 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[24488]: audit 2024-02-10T08:44:38.949494+0000 mon.c (mon.2) 1047 : audit [INF] from='client.? 172.21.15.67:0/2736237525' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:39.871 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:39 smithi067 bash[24488]: audit 2024-02-10T08:44:38.950133+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:40.026 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:39 smithi134 bash[18722]: cluster 2024-02-10T08:44:38.434669+0000 mgr.y (mgr.14547) 844 : cluster [DBG] pgmap v1333: 188 pgs: 188 active+clean; 298 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-10T08:44:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:39 smithi134 bash[18722]: audit 2024-02-10T08:44:38.556859+0000 mon.a (mon.0) 3217 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]': finished 2024-02-10T08:44:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:39 smithi134 bash[18722]: cluster 2024-02-10T08:44:38.557060+0000 mon.a (mon.0) 3218 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-10T08:44:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:39 smithi134 bash[18722]: audit 2024-02-10T08:44:38.660471+0000 mon.a (mon.0) 3219 : audit [INF] from='client.? 172.21.15.67:0/2791051801' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "34c11e82-d78d-4916-9c14-acce32e59950", "pool2": "34c11e82-d78d-4916-9c14-acce32e59950", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:39 smithi134 bash[18722]: audit 2024-02-10T08:44:38.949494+0000 mon.c (mon.2) 1047 : audit [INF] from='client.? 172.21.15.67:0/2736237525' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:40.027 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:39 smithi134 bash[18722]: audit 2024-02-10T08:44:38.950133+0000 mon.a (mon.0) 3220 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "pool2": "1b373566-6b27-4e28-8afd-e7e8ebc4bc37", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-10T08:44:40.053 DEBUG:teuthology.parallel:result is None 2024-02-10T08:44:40.053 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T08:44:40.066 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T08:44:40.067 DEBUG:teuthology.orchestra.run.smithi067:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-10T08:44:40.128 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-10T08:44:40.129 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-10T08:44:40.141 INFO:tasks.cephadm:Teardown begin 2024-02-10T08:44:40.141 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:44:40.186 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:44:40.202 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-10T08:44:40.202 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T08:44:40.236 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T08:44:40.252 INFO:tasks.cephadm:Stopping all daemons... 2024-02-10T08:44:40.252 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-10T08:44:40.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a 2024-02-10T08:44:40.302 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:40 smithi067 systemd[1]: Stopping Ceph mon.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:40.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:40 smithi067 bash[121472]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mon.a 2024-02-10T08:44:40.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:40 smithi067 bash[17385]: debug 2024-02-10T08:44:40.526+0000 7f94d0a6f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-10T08:44:40.614 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 10 08:44:40 smithi067 bash[17385]: debug 2024-02-10T08:44:40.526+0000 7f94d0a6f700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-10T08:44:41.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.a.service' 2024-02-10T08:44:41.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:44:41.825 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-10T08:44:41.825 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-10T08:44:41.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.c 2024-02-10T08:44:42.107 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:41 smithi067 systemd[1]: Stopping Ceph mon.c for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:41 smithi067 bash[121592]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mon.c 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[24488]: debug 2024-02-10T08:44:42.030+0000 7f0f47583700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mon.c.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[24488]: debug 2024-02-10T08:44:42.030+0000 7f0f47583700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[17647]: [10/Feb/2024:08:44:42] ENGINE Bus STOPPING 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[17647]: [10/Feb/2024:08:44:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[17647]: [10/Feb/2024:08:44:42] ENGINE Bus STOPPED 2024-02-10T08:44:42.108 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[17647]: [10/Feb/2024:08:44:42] ENGINE Bus STARTING 2024-02-10T08:44:42.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[17647]: [10/Feb/2024:08:44:42] ENGINE Serving on http://:::9283 2024-02-10T08:44:42.370 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:42 smithi067 bash[17647]: [10/Feb/2024:08:44:42] ENGINE Bus STARTED 2024-02-10T08:44:42.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.c.service' 2024-02-10T08:44:42.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:44:42.903 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-10T08:44:42.903 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-10T08:44:42.903 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.b 2024-02-10T08:44:42.945 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:44:42 smithi134 bash[39502]: debug there is no tcmu-runner data available 2024-02-10T08:44:42.945 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:42 smithi134 systemd[1]: Stopping Ceph mon.b for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:43.245 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:43 smithi134 bash[19742]: ::ffff:172.21.15.134 - - [10/Feb/2024:08:44:43] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-10T08:44:43.246 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:42 smithi134 bash[44323]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mon.b 2024-02-10T08:44:43.246 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:43 smithi134 bash[18722]: debug 2024-02-10T08:44:43.176+0000 7f324ca60700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-10T08:44:43.246 INFO:journalctl@ceph.mon.b.smithi134.stdout:Feb 10 08:44:43 smithi134 bash[18722]: debug 2024-02-10T08:44:43.176+0000 7f324ca60700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-10T08:44:44.126 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mon.b.service' 2024-02-10T08:44:44.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:44:44.215 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-10T08:44:44.215 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-10T08:44:44.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.y 2024-02-10T08:44:44.482 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:44 smithi067 systemd[1]: Stopping Ceph mgr.y for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:44.482 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 10 08:44:44 smithi067 bash[121720]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mgr.y 2024-02-10T08:44:45.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.y.service' 2024-02-10T08:44:45.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:44:45.303 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-10T08:44:45.303 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-10T08:44:45.303 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.x 2024-02-10T08:44:45.569 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:45 smithi134 systemd[1]: Stopping Ceph mgr.x for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:45.570 INFO:journalctl@ceph.mgr.x.smithi134.stdout:Feb 10 08:44:45 smithi134 bash[44442]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-mgr.x 2024-02-10T08:44:46.259 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@mgr.x.service' 2024-02-10T08:44:46.295 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:44:46.296 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-10T08:44:46.296 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-10T08:44:46.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.0 2024-02-10T08:44:46.620 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:46 smithi067 systemd[1]: Stopping Ceph osd.0 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:46.620 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:46 smithi067 bash[121835]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.0 2024-02-10T08:44:46.620 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:46 smithi067 bash[27635]: debug 2024-02-10T08:44:46.494+0000 7ff65bd16700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:44:46.620 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:46 smithi067 bash[27635]: debug 2024-02-10T08:44:46.494+0000 7ff65bd16700 -1 osd.0 743 *** Got signal Terminated *** 2024-02-10T08:44:46.620 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:46 smithi067 bash[27635]: debug 2024-02-10T08:44:46.494+0000 7ff65bd16700 -1 osd.0 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:44:50.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:44:50 smithi067 bash[48202]: level=warn ts=2024-02-10T08:44:50.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:44:50.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:44:50 smithi067 bash[48202]: level=warn ts=2024-02-10T08:44:50.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:44:52.480 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:52 smithi067 bash[121869]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-0 2024-02-10T08:44:52.480 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 10 08:44:52 smithi067 bash[121939]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-0 2024-02-10T08:44:53.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:44:53 smithi067 bash[48202]: level=warn ts=2024-02-10T08:44:53.571Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:44:53.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:44:53 smithi067 bash[48202]: level=warn ts=2024-02-10T08:44:53.572Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:44:54.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.0.service' 2024-02-10T08:44:54.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:44:54.630 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-10T08:44:54.630 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-10T08:44:54.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.1 2024-02-10T08:44:54.867 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:44:54 smithi067 systemd[1]: Stopping Ceph osd.1 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:44:54.867 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:44:54 smithi067 bash[122056]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.1 2024-02-10T08:44:55.120 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:44:54 smithi067 bash[32341]: debug 2024-02-10T08:44:54.862+0000 7fe8d1e0f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:44:55.120 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:44:54 smithi067 bash[32341]: debug 2024-02-10T08:44:54.862+0000 7fe8d1e0f700 -1 osd.1 743 *** Got signal Terminated *** 2024-02-10T08:44:55.120 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:44:54 smithi067 bash[32341]: debug 2024-02-10T08:44:54.862+0000 7fe8d1e0f700 -1 osd.1 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:00.370 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:00 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:00.145Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:00.370 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:00 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:00.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:00.370 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:00 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:00.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:00.864 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:45:00 smithi067 bash[122093]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-1 2024-02-10T08:45:00.864 INFO:journalctl@ceph.osd.1.smithi067.stdout:Feb 10 08:45:00 smithi067 bash[122157]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-1 2024-02-10T08:45:02.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.1.service' 2024-02-10T08:45:02.814 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:02.814 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-10T08:45:02.814 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-10T08:45:02.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.2 2024-02-10T08:45:03.048 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:02 smithi067 systemd[1]: Stopping Ceph osd.2 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:03.048 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:02 smithi067 bash[122277]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.2 2024-02-10T08:45:03.370 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:03 smithi067 bash[37094]: debug 2024-02-10T08:45:03.046+0000 7f035b387700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:03.370 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:03 smithi067 bash[37094]: debug 2024-02-10T08:45:03.046+0000 7f035b387700 -1 osd.2 743 *** Got signal Terminated *** 2024-02-10T08:45:03.370 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:03 smithi067 bash[37094]: debug 2024-02-10T08:45:03.046+0000 7f035b387700 -1 osd.2 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:03.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:03 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:03.571Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:03.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:03 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:03.572Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:03.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:03 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:03.573Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:09.120 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:08 smithi067 bash[122313]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-2 2024-02-10T08:45:09.120 INFO:journalctl@ceph.osd.2.smithi067.stdout:Feb 10 08:45:08 smithi067 bash[122382]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-2 2024-02-10T08:45:10.519 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:10 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:10.146Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:10.519 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:10 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:10.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:10.519 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:10 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:10.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:10.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.2.service' 2024-02-10T08:45:11.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:11.024 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-10T08:45:11.024 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-10T08:45:11.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.3 2024-02-10T08:45:11.272 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:11 smithi067 systemd[1]: Stopping Ceph osd.3 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:11.272 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:11 smithi067 bash[122497]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.3 2024-02-10T08:45:11.619 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:11 smithi067 bash[41706]: debug 2024-02-10T08:45:11.270+0000 7f177f14f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:11.620 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:11 smithi067 bash[41706]: debug 2024-02-10T08:45:11.270+0000 7f177f14f700 -1 osd.3 743 *** Got signal Terminated *** 2024-02-10T08:45:11.620 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:11 smithi067 bash[41706]: debug 2024-02-10T08:45:11.270+0000 7f177f14f700 -1 osd.3 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:12.620 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:12 smithi067 bash[41706]: debug 2024-02-10T08:45:12.126+0000 7f177a0c8700 -1 osd.3 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.221556+0000 front 2024-02-10T08:44:49.221929+0000 (oldest deadline 2024-02-10T08:45:12.121666+0000) 2024-02-10T08:45:13.026 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:12 smithi134 bash[22457]: debug 2024-02-10T08:45:12.548+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:13.526 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:13 smithi134 bash[36245]: debug 2024-02-10T08:45:13.227+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:13.572 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:13 smithi067 bash[41706]: debug 2024-02-10T08:45:13.122+0000 7f177a0c8700 -1 osd.3 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.221556+0000 front 2024-02-10T08:44:49.221929+0000 (oldest deadline 2024-02-10T08:45:12.121666+0000) 2024-02-10T08:45:13.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:13 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:13.571Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:13.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:13 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:13.572Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:13.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:13 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:13.574Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:14.026 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:13 smithi134 bash[22457]: debug 2024-02-10T08:45:13.547+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:14.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:13 smithi134 bash[31507]: debug 2024-02-10T08:45:13.783+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:14.370 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:14 smithi067 bash[41706]: debug 2024-02-10T08:45:14.078+0000 7f177a0c8700 -1 osd.3 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.221556+0000 front 2024-02-10T08:44:49.221929+0000 (oldest deadline 2024-02-10T08:45:12.121666+0000) 2024-02-10T08:45:14.526 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:14 smithi134 bash[26946]: debug 2024-02-10T08:45:14.347+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:14.526 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:14 smithi134 bash[36245]: debug 2024-02-10T08:45:14.179+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:15.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:14 smithi134 bash[31507]: debug 2024-02-10T08:45:14.763+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:15.026 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:14 smithi134 bash[22457]: debug 2024-02-10T08:45:14.583+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:15.370 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:15 smithi067 bash[41706]: debug 2024-02-10T08:45:15.078+0000 7f177a0c8700 -1 osd.3 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.221556+0000 front 2024-02-10T08:44:49.221929+0000 (oldest deadline 2024-02-10T08:45:12.121666+0000) 2024-02-10T08:45:15.526 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:15 smithi134 bash[26946]: debug 2024-02-10T08:45:15.351+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:15.526 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:15 smithi134 bash[36245]: debug 2024-02-10T08:45:15.139+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:16.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:15 smithi134 bash[31507]: debug 2024-02-10T08:45:15.755+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:16.027 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:15 smithi134 bash[22457]: debug 2024-02-10T08:45:15.607+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:16.367 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:16 smithi134 bash[36245]: debug 2024-02-10T08:45:16.111+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:16.381 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:16 smithi067 bash[41706]: debug 2024-02-10T08:45:16.126+0000 7f177a0c8700 -1 osd.3 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.221556+0000 front 2024-02-10T08:44:49.221929+0000 (oldest deadline 2024-02-10T08:45:12.121666+0000) 2024-02-10T08:45:16.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:16 smithi134 bash[26946]: debug 2024-02-10T08:45:16.363+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:16.777 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:16 smithi134 bash[22457]: debug 2024-02-10T08:45:16.599+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:17.104 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:16 smithi134 bash[31507]: debug 2024-02-10T08:45:16.783+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:17.119 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:16 smithi067 bash[122532]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-3 2024-02-10T08:45:17.119 INFO:journalctl@ceph.osd.3.smithi067.stdout:Feb 10 08:45:17 smithi067 bash[122598]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-3 2024-02-10T08:45:17.368 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:17 smithi134 bash[36245]: debug 2024-02-10T08:45:17.103+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:17.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:17 smithi134 bash[22457]: debug 2024-02-10T08:45:17.563+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:17.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:17 smithi134 bash[26946]: debug 2024-02-10T08:45:17.363+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:18.276 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:17 smithi134 bash[31507]: debug 2024-02-10T08:45:17.819+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:18.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:18 smithi134 bash[36245]: debug 2024-02-10T08:45:18.055+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:18.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:18 smithi134 bash[22457]: debug 2024-02-10T08:45:18.519+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:18.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:18 smithi134 bash[26946]: debug 2024-02-10T08:45:18.387+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:19.105 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:18 smithi134 bash[31507]: debug 2024-02-10T08:45:18.839+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:19.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.3.service' 2024-02-10T08:45:19.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:19.146 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-10T08:45:19.146 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-10T08:45:19.146 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.4 2024-02-10T08:45:19.383 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[36245]: debug 2024-02-10T08:45:19.103+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:19.383 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:19 smithi134 systemd[1]: Stopping Ceph osd.4 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:19.383 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[44557]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.4 2024-02-10T08:45:19.383 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[22457]: debug 2024-02-10T08:45:19.323+0000 7f230bb5f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:19.383 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[22457]: debug 2024-02-10T08:45:19.323+0000 7f230bb5f700 -1 osd.4 743 *** Got signal Terminated *** 2024-02-10T08:45:19.383 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[22457]: debug 2024-02-10T08:45:19.323+0000 7f230bb5f700 -1 osd.4 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:19.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[26946]: debug 2024-02-10T08:45:19.379+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:19.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[22457]: debug 2024-02-10T08:45:19.555+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:20.098 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:19 smithi134 bash[31507]: debug 2024-02-10T08:45:19.827+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:20.397 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:20 smithi134 bash[36245]: debug 2024-02-10T08:45:20.091+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:20.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:20 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:20.147Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:20.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:20 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:20.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:20.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:20 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:20.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:20.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:20 smithi134 bash[26946]: debug 2024-02-10T08:45:20.391+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:20.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:20 smithi134 bash[22457]: debug 2024-02-10T08:45:20.579+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:20.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:20 smithi134 bash[22457]: debug 2024-02-10T08:45:20.579+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.546819+0000 front 2024-02-10T08:44:57.547072+0000 (oldest deadline 2024-02-10T08:45:20.446648+0000) 2024-02-10T08:45:21.276 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:20 smithi134 bash[31507]: debug 2024-02-10T08:45:20.827+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:21.276 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:20 smithi134 bash[31507]: debug 2024-02-10T08:45:20.827+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:21.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:21 smithi134 bash[36245]: debug 2024-02-10T08:45:21.047+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:21.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:21 smithi134 bash[26946]: debug 2024-02-10T08:45:21.439+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:21.777 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:21 smithi134 bash[22457]: debug 2024-02-10T08:45:21.619+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:21.777 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:21 smithi134 bash[22457]: debug 2024-02-10T08:45:21.619+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.546819+0000 front 2024-02-10T08:44:57.547072+0000 (oldest deadline 2024-02-10T08:45:20.446648+0000) 2024-02-10T08:45:22.089 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:21 smithi134 bash[31507]: debug 2024-02-10T08:45:21.827+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:22.089 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:21 smithi134 bash[31507]: debug 2024-02-10T08:45:21.827+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:22.444 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:22 smithi134 bash[36245]: debug 2024-02-10T08:45:22.087+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:22.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:22 smithi134 bash[26946]: debug 2024-02-10T08:45:22.443+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:22.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:22 smithi134 bash[22457]: debug 2024-02-10T08:45:22.663+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:22.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:22 smithi134 bash[22457]: debug 2024-02-10T08:45:22.663+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.546819+0000 front 2024-02-10T08:44:57.547072+0000 (oldest deadline 2024-02-10T08:45:20.446648+0000) 2024-02-10T08:45:23.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[36245]: debug 2024-02-10T08:45:23.047+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:23.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[36245]: debug 2024-02-10T08:45:23.047+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:23.276 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:22 smithi134 bash[31507]: debug 2024-02-10T08:45:22.811+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:23.277 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:22 smithi134 bash[31507]: debug 2024-02-10T08:45:22.811+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:23.776 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[22457]: debug 2024-02-10T08:45:23.703+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.045033+0000 front 2024-02-10T08:44:50.045328+0000 (oldest deadline 2024-02-10T08:45:11.744881+0000) 2024-02-10T08:45:23.777 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[22457]: debug 2024-02-10T08:45:23.703+0000 7f2308adc700 -1 osd.4 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.546819+0000 front 2024-02-10T08:44:57.547072+0000 (oldest deadline 2024-02-10T08:45:20.446648+0000) 2024-02-10T08:45:23.777 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[26946]: debug 2024-02-10T08:45:23.455+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:23.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:23 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:23.572Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:23.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:23 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:23.573Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:23.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:23 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:23.574Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:24.061 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[31507]: debug 2024-02-10T08:45:23.791+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:24.061 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:23 smithi134 bash[31507]: debug 2024-02-10T08:45:23.791+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:24.411 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:24 smithi134 bash[36245]: debug 2024-02-10T08:45:24.059+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:24.411 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:24 smithi134 bash[36245]: debug 2024-02-10T08:45:24.059+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:24.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:24 smithi134 bash[26946]: debug 2024-02-10T08:45:24.407+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:25.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:24 smithi134 bash[31507]: debug 2024-02-10T08:45:24.815+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:25.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:24 smithi134 bash[31507]: debug 2024-02-10T08:45:24.815+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:25.349 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[36245]: debug 2024-02-10T08:45:25.079+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:25.349 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[36245]: debug 2024-02-10T08:45:25.079+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:25.349 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[44592]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-4 2024-02-10T08:45:25.350 INFO:journalctl@ceph.osd.4.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[44658]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-4 2024-02-10T08:45:25.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[26946]: debug 2024-02-10T08:45:25.455+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:25.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[26946]: debug 2024-02-10T08:45:25.455+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:26.255 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[31507]: debug 2024-02-10T08:45:25.819+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:26.255 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:25 smithi134 bash[31507]: debug 2024-02-10T08:45:25.819+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:26.255 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:26 smithi134 bash[36245]: debug 2024-02-10T08:45:26.063+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:26.255 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:26 smithi134 bash[36245]: debug 2024-02-10T08:45:26.063+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:26.526 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:26 smithi134 bash[26946]: debug 2024-02-10T08:45:26.479+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:26.526 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:26 smithi134 bash[26946]: debug 2024-02-10T08:45:26.479+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:27.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:26 smithi134 bash[31507]: debug 2024-02-10T08:45:26.787+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:27.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:26 smithi134 bash[31507]: debug 2024-02-10T08:45:26.787+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:27.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[36245]: debug 2024-02-10T08:45:27.071+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:27.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[36245]: debug 2024-02-10T08:45:27.071+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:27.362 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.4.service' 2024-02-10T08:45:27.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:27.388 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-10T08:45:27.388 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-10T08:45:27.389 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.5 2024-02-10T08:45:27.628 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 systemd[1]: Stopping Ceph osd.5 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:27.628 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[44775]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.5 2024-02-10T08:45:27.629 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[26946]: debug 2024-02-10T08:45:27.495+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:27.629 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[26946]: debug 2024-02-10T08:45:27.495+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:27.629 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[26946]: debug 2024-02-10T08:45:27.623+0000 7f86aba88700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:28.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[31507]: debug 2024-02-10T08:45:27.739+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:28.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[31507]: debug 2024-02-10T08:45:27.739+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:28.026 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[26946]: debug 2024-02-10T08:45:27.623+0000 7f86aba88700 -1 osd.5 743 *** Got signal Terminated *** 2024-02-10T08:45:28.027 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:27 smithi134 bash[26946]: debug 2024-02-10T08:45:27.623+0000 7f86aba88700 -1 osd.5 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:28.524 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:28 smithi134 bash[36245]: debug 2024-02-10T08:45:28.027+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:28.524 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:28 smithi134 bash[36245]: debug 2024-02-10T08:45:28.027+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:28.776 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:28 smithi134 bash[31507]: debug 2024-02-10T08:45:28.759+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:28.776 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:28 smithi134 bash[31507]: debug 2024-02-10T08:45:28.759+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:28.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:28 smithi134 bash[26946]: debug 2024-02-10T08:45:28.519+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:28.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:28 smithi134 bash[26946]: debug 2024-02-10T08:45:28.519+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:29.526 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[36245]: debug 2024-02-10T08:45:29.031+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:29.526 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[36245]: debug 2024-02-10T08:45:29.031+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:30.006 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[26946]: debug 2024-02-10T08:45:29.559+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:30.006 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[26946]: debug 2024-02-10T08:45:29.559+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:30.006 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[31507]: debug 2024-02-10T08:45:29.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:30.007 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[31507]: debug 2024-02-10T08:45:29.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:30.007 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:29 smithi134 bash[31507]: debug 2024-02-10T08:45:29.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:30.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[36245]: debug 2024-02-10T08:45:29.999+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:30.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[36245]: debug 2024-02-10T08:45:29.999+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:30.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:30 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:30.148Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:30.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:30 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:30.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:30.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:30 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:30.150Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:30.979 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[26946]: debug 2024-02-10T08:45:30.571+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:30.980 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[26946]: debug 2024-02-10T08:45:30.571+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:30.980 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[31507]: debug 2024-02-10T08:45:30.755+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:30.980 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[31507]: debug 2024-02-10T08:45:30.755+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:30.980 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[31507]: debug 2024-02-10T08:45:30.755+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:31.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[36245]: debug 2024-02-10T08:45:30.975+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:31.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:30 smithi134 bash[36245]: debug 2024-02-10T08:45:30.975+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:31.984 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[26946]: debug 2024-02-10T08:45:31.591+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:31.984 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[26946]: debug 2024-02-10T08:45:31.591+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:31.984 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[31507]: debug 2024-02-10T08:45:31.767+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:31.984 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[31507]: debug 2024-02-10T08:45:31.767+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:31.984 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[31507]: debug 2024-02-10T08:45:31.767+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:32.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[36245]: debug 2024-02-10T08:45:31.979+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:32.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:31 smithi134 bash[36245]: debug 2024-02-10T08:45:31.979+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:32.900 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:32 smithi134 bash[26946]: debug 2024-02-10T08:45:32.547+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:47.639670+0000 front 2024-02-10T08:44:47.639386+0000 (oldest deadline 2024-02-10T08:45:13.539489+0000) 2024-02-10T08:45:32.900 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:32 smithi134 bash[26946]: debug 2024-02-10T08:45:32.547+0000 7f86a59ff700 -1 osd.5 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:59.341992+0000 front 2024-02-10T08:44:59.342055+0000 (oldest deadline 2024-02-10T08:45:25.242089+0000) 2024-02-10T08:45:32.900 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:32 smithi134 bash[31507]: debug 2024-02-10T08:45:32.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:32.900 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:32 smithi134 bash[31507]: debug 2024-02-10T08:45:32.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:32.900 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:32 smithi134 bash[31507]: debug 2024-02-10T08:45:32.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:33.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[36245]: debug 2024-02-10T08:45:32.987+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:33.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[36245]: debug 2024-02-10T08:45:32.987+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:33.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[44809]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-5 2024-02-10T08:45:33.776 INFO:journalctl@ceph.osd.5.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[44874]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-5 2024-02-10T08:45:33.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:33 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:33.573Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:33.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:33 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:33.574Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:33.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:33 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:33.575Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:34.260 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[31507]: debug 2024-02-10T08:45:33.771+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:34.261 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[31507]: debug 2024-02-10T08:45:33.771+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:34.261 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[31507]: debug 2024-02-10T08:45:33.771+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:34.261 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[36245]: debug 2024-02-10T08:45:33.983+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:34.261 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[36245]: debug 2024-02-10T08:45:33.983+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:34.261 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:33 smithi134 bash[36245]: debug 2024-02-10T08:45:33.983+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:35.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:34 smithi134 bash[31507]: debug 2024-02-10T08:45:34.787+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:35.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:34 smithi134 bash[31507]: debug 2024-02-10T08:45:34.787+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:35.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:34 smithi134 bash[31507]: debug 2024-02-10T08:45:34.787+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:35.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:34 smithi134 bash[36245]: debug 2024-02-10T08:45:34.955+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:35.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:34 smithi134 bash[36245]: debug 2024-02-10T08:45:34.955+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:35.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:34 smithi134 bash[36245]: debug 2024-02-10T08:45:34.955+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:35.450 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.5.service' 2024-02-10T08:45:35.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:35.480 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-10T08:45:35.480 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-10T08:45:35.480 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.6 2024-02-10T08:45:35.728 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 systemd[1]: Stopping Ceph osd.6 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:35.728 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[44991]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.6 2024-02-10T08:45:35.981 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[31507]: debug 2024-02-10T08:45:35.723+0000 7fcba4140700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:35.982 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[31507]: debug 2024-02-10T08:45:35.723+0000 7fcba4140700 -1 osd.6 743 *** Got signal Terminated *** 2024-02-10T08:45:35.982 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[31507]: debug 2024-02-10T08:45:35.723+0000 7fcba4140700 -1 osd.6 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:35.982 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[31507]: debug 2024-02-10T08:45:35.807+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:35.982 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[31507]: debug 2024-02-10T08:45:35.807+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:35.982 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[31507]: debug 2024-02-10T08:45:35.807+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:36.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[36245]: debug 2024-02-10T08:45:35.975+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:36.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[36245]: debug 2024-02-10T08:45:35.975+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:36.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:35 smithi134 bash[36245]: debug 2024-02-10T08:45:35.975+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:37.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:36 smithi134 bash[31507]: debug 2024-02-10T08:45:36.759+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:37.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:36 smithi134 bash[31507]: debug 2024-02-10T08:45:36.759+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:37.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:36 smithi134 bash[31507]: debug 2024-02-10T08:45:36.759+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:37.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:36 smithi134 bash[36245]: debug 2024-02-10T08:45:36.995+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:37.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:36 smithi134 bash[36245]: debug 2024-02-10T08:45:36.995+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:37.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:36 smithi134 bash[36245]: debug 2024-02-10T08:45:36.995+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:38.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[31507]: debug 2024-02-10T08:45:37.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:38.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[31507]: debug 2024-02-10T08:45:37.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:38.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[31507]: debug 2024-02-10T08:45:37.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:38.027 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[31507]: debug 2024-02-10T08:45:37.727+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:14.399207+0000 front 2024-02-10T08:45:14.399244+0000 (oldest deadline 2024-02-10T08:45:37.299043+0000) 2024-02-10T08:45:38.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[36245]: debug 2024-02-10T08:45:37.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:38.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[36245]: debug 2024-02-10T08:45:37.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:38.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[36245]: debug 2024-02-10T08:45:37.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:38.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:37 smithi134 bash[36245]: debug 2024-02-10T08:45:37.951+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:39.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[31507]: debug 2024-02-10T08:45:38.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:39.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[31507]: debug 2024-02-10T08:45:38.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:39.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[31507]: debug 2024-02-10T08:45:38.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:39.027 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[31507]: debug 2024-02-10T08:45:38.719+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:14.399207+0000 front 2024-02-10T08:45:14.399244+0000 (oldest deadline 2024-02-10T08:45:37.299043+0000) 2024-02-10T08:45:39.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[36245]: debug 2024-02-10T08:45:38.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:39.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[36245]: debug 2024-02-10T08:45:38.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:39.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[36245]: debug 2024-02-10T08:45:38.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:39.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[36245]: debug 2024-02-10T08:45:38.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:39.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:38 smithi134 bash[36245]: debug 2024-02-10T08:45:38.951+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:40.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[31507]: debug 2024-02-10T08:45:39.675+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:40.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[31507]: debug 2024-02-10T08:45:39.675+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:40.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[31507]: debug 2024-02-10T08:45:39.675+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:40.026 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[31507]: debug 2024-02-10T08:45:39.675+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:14.399207+0000 front 2024-02-10T08:45:14.399244+0000 (oldest deadline 2024-02-10T08:45:37.299043+0000) 2024-02-10T08:45:40.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[36245]: debug 2024-02-10T08:45:39.911+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:40.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[36245]: debug 2024-02-10T08:45:39.911+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:40.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[36245]: debug 2024-02-10T08:45:39.911+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:40.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[36245]: debug 2024-02-10T08:45:39.911+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:40.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:39 smithi134 bash[36245]: debug 2024-02-10T08:45:39.911+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:40.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:40 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:40.149Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:40.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:40 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:40.150Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:40.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:40 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:40.151Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:41.023 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[36245]: debug 2024-02-10T08:45:40.879+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[36245]: debug 2024-02-10T08:45:40.879+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[36245]: debug 2024-02-10T08:45:40.879+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[36245]: debug 2024-02-10T08:45:40.879+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[36245]: debug 2024-02-10T08:45:40.879+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[31507]: debug 2024-02-10T08:45:40.699+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:50.093504+0000 front 2024-02-10T08:44:50.093188+0000 (oldest deadline 2024-02-10T08:45:12.993270+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[31507]: debug 2024-02-10T08:45:40.699+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:56.995612+0000 front 2024-02-10T08:44:56.995667+0000 (oldest deadline 2024-02-10T08:45:20.495136+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[31507]: debug 2024-02-10T08:45:40.699+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:04.497069+0000 front 2024-02-10T08:45:04.497090+0000 (oldest deadline 2024-02-10T08:45:29.197011+0000) 2024-02-10T08:45:41.024 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:40 smithi134 bash[31507]: debug 2024-02-10T08:45:40.699+0000 7fcba10bd700 -1 osd.6 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:14.399207+0000 front 2024-02-10T08:45:14.399244+0000 (oldest deadline 2024-02-10T08:45:37.299043+0000) 2024-02-10T08:45:41.776 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[45027]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-6 2024-02-10T08:45:41.776 INFO:journalctl@ceph.osd.6.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[45092]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-6 2024-02-10T08:45:42.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[36245]: debug 2024-02-10T08:45:41.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:42.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[36245]: debug 2024-02-10T08:45:41.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:42.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[36245]: debug 2024-02-10T08:45:41.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:42.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[36245]: debug 2024-02-10T08:45:41.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:42.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:41 smithi134 bash[36245]: debug 2024-02-10T08:45:41.895+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:43.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:42 smithi134 bash[36245]: debug 2024-02-10T08:45:42.891+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:43.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:42 smithi134 bash[36245]: debug 2024-02-10T08:45:42.891+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:43.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:42 smithi134 bash[36245]: debug 2024-02-10T08:45:42.891+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:43.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:42 smithi134 bash[36245]: debug 2024-02-10T08:45:42.891+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:43.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:42 smithi134 bash[36245]: debug 2024-02-10T08:45:42.891+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:43.446 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.6.service' 2024-02-10T08:45:43.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:43.475 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-10T08:45:43.476 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-10T08:45:43.476 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.7 2024-02-10T08:45:43.728 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 systemd[1]: Stopping Ceph osd.7 for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:43.728 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[45207]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd.7 2024-02-10T08:45:43.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:43 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:43.574Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:43.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:43 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:43.575Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:43.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:43 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:43.576Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.723+0000 7f6514909700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.723+0000 7f6514909700 -1 osd.7 743 *** Got signal Terminated *** 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.723+0000 7f6514909700 -1 osd.7 743 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.931+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.931+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.931+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:44.026 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.931+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:44.027 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:43 smithi134 bash[36245]: debug 2024-02-10T08:45:43.931+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:45.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:44 smithi134 bash[36245]: debug 2024-02-10T08:45:44.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:45.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:44 smithi134 bash[36245]: debug 2024-02-10T08:45:44.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:45.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:44 smithi134 bash[36245]: debug 2024-02-10T08:45:44.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:45.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:44 smithi134 bash[36245]: debug 2024-02-10T08:45:44.951+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:45.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:44 smithi134 bash[36245]: debug 2024-02-10T08:45:44.951+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:46.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:45 smithi134 bash[36245]: debug 2024-02-10T08:45:45.915+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:46.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:45 smithi134 bash[36245]: debug 2024-02-10T08:45:45.915+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:46.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:45 smithi134 bash[36245]: debug 2024-02-10T08:45:45.915+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:46.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:45 smithi134 bash[36245]: debug 2024-02-10T08:45:45.915+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:46.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:45 smithi134 bash[36245]: debug 2024-02-10T08:45:45.915+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:47.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:46 smithi134 bash[36245]: debug 2024-02-10T08:45:46.907+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:47.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:46 smithi134 bash[36245]: debug 2024-02-10T08:45:46.907+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:47.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:46 smithi134 bash[36245]: debug 2024-02-10T08:45:46.907+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:47.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:46 smithi134 bash[36245]: debug 2024-02-10T08:45:46.907+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:47.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:46 smithi134 bash[36245]: debug 2024-02-10T08:45:46.907+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.134:6804 osd.4 since back 2024-02-10T08:45:23.589051+0000 front 2024-02-10T08:45:23.588993+0000 (oldest deadline 2024-02-10T08:45:46.488851+0000) 2024-02-10T08:45:47.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:46 smithi134 bash[36245]: debug 2024-02-10T08:45:46.907+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:48.276 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:47 smithi134 bash[36245]: debug 2024-02-10T08:45:47.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6806 osd.0 since back 2024-02-10T08:44:49.583847+0000 front 2024-02-10T08:44:49.583610+0000 (oldest deadline 2024-02-10T08:45:13.083631+0000) 2024-02-10T08:45:48.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:47 smithi134 bash[36245]: debug 2024-02-10T08:45:47.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6814 osd.1 since back 2024-02-10T08:44:57.685787+0000 front 2024-02-10T08:44:57.685825+0000 (oldest deadline 2024-02-10T08:45:22.385586+0000) 2024-02-10T08:45:48.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:47 smithi134 bash[36245]: debug 2024-02-10T08:45:47.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6822 osd.2 since back 2024-02-10T08:45:07.686935+0000 front 2024-02-10T08:45:07.687238+0000 (oldest deadline 2024-02-10T08:45:33.586974+0000) 2024-02-10T08:45:48.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:47 smithi134 bash[36245]: debug 2024-02-10T08:45:47.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.67:6830 osd.3 since back 2024-02-10T08:45:13.587723+0000 front 2024-02-10T08:45:13.587846+0000 (oldest deadline 2024-02-10T08:45:38.887555+0000) 2024-02-10T08:45:48.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:47 smithi134 bash[36245]: debug 2024-02-10T08:45:47.895+0000 7f6511886700 -1 osd.7 743 heartbeat_check: no reply from 172.21.15.134:6804 osd.4 since back 2024-02-10T08:45:23.589051+0000 front 2024-02-10T08:45:23.588993+0000 (oldest deadline 2024-02-10T08:45:46.488851+0000) 2024-02-10T08:45:48.277 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:47 smithi134 bash[36245]: debug 2024-02-10T08:45:47.895+0000 7f6511886700 -1 osd.7 743 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24436.0:1335 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e743) 2024-02-10T08:45:49.576 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:49 smithi134 bash[45245]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-7 2024-02-10T08:45:49.576 INFO:journalctl@ceph.osd.7.smithi134.stdout:Feb 10 08:45:49 smithi134 bash[45310]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-osd-7 2024-02-10T08:45:50.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:50 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:50.150Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:50.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:50 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:50.151Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:50.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:50 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:50.152Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:45:51.376 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@osd.7.service' 2024-02-10T08:45:51.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:45:51.407 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-10T08:45:51.407 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-10T08:45:51.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@rgw.foo.a 2024-02-10T08:45:51.870 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 10 08:45:51 smithi067 systemd[1]: Stopping Ceph rgw.foo.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:45:51.870 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 10 08:45:51 smithi067 bash[122715]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-rgw.foo.a 2024-02-10T08:45:51.870 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 10 08:45:51 smithi067 bash[44225]: debug 2024-02-10T08:45:51.605+0000 7fce65326700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-10T08:45:51.870 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 10 08:45:51 smithi067 bash[44225]: debug 2024-02-10T08:45:51.605+0000 7fce7c3f0880 -1 shutting down 2024-02-10T08:45:53.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:53 smithi067 bash[48202]: level=error ts=2024-02-10T08:45:53.575Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:53.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:53 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:53.576Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:45:53.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:45:53 smithi067 bash[48202]: level=warn ts=2024-02-10T08:45:53.577Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:46:00.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:00 smithi067 bash[48202]: level=error ts=2024-02-10T08:46:00.151Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:00.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:00 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:00.152Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:00.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:00 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:00.153Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:46:01.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:01 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:01.625Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:01.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:01 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:01.625Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:46:02.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@rgw.foo.a.service' 2024-02-10T08:46:02.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:46:02.373 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-10T08:46:02.373 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-10T08:46:02.373 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@prometheus.a 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 systemd[1]: Stopping Ceph prometheus.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[45423]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-prometheus.a 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.567Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.567Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.567Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.567Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-10T08:46:02.771 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.567Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.568Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.568Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.568Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.568Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.569Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.569Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-10T08:46:02.772 INFO:journalctl@ceph.prometheus.a.smithi134.stdout:Feb 10 08:46:02 smithi134 bash[42505]: ts=2024-02-10T08:46:02.569Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-10T08:46:03.238 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@prometheus.a.service' 2024-02-10T08:46:03.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T08:46:03.268 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-10T08:46:03.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm rm-cluster --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T08:46:03.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:03 smithi067 bash[48202]: level=error ts=2024-02-10T08:46:03.576Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:03.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:03 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:03.577Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:03.870 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:03 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:03.578Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:46:10.120 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:09 smithi067 systemd[1]: Stopping Ceph node-exporter.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:46:10.120 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:09 smithi067 bash[123139]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-node-exporter.a 2024-02-10T08:46:10.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:10 smithi067 bash[48202]: level=error ts=2024-02-10T08:46:10.152Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:10.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:10 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:10.153Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi067.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.67:8443: connect: connection refused" 2024-02-10T08:46:10.620 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:10 smithi067 bash[48202]: level=warn ts=2024-02-10T08:46:10.154Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi134.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.134:8443: connect: connection refused" 2024-02-10T08:46:11.092 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:10 smithi067 bash[123176]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-node-exporter-a 2024-02-10T08:46:11.093 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:10 smithi067 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T08:46:11.093 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:10 smithi067 bash[123237]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-node-exporter-a 2024-02-10T08:46:11.093 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:10 smithi067 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-10T08:46:11.093 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 10 08:46:10 smithi067 systemd[1]: Stopped Ceph node-exporter.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:46:11.349 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:11 smithi067 systemd[1]: Stopping Ceph alertmanager.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:46:11.349 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:11 smithi067 bash[123273]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-alertmanager.a 2024-02-10T08:46:11.349 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:11 smithi067 bash[48202]: level=info ts=2024-02-10T08:46:11.275Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-10T08:46:12.313 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:11 smithi067 bash[123308]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T08:46:12.313 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:12 smithi067 bash[123369]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T08:46:12.313 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:12 smithi067 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@alertmanager.a.service: Succeeded. 2024-02-10T08:46:12.313 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 10 08:46:12 smithi067 systemd[1]: Stopped Ceph alertmanager.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:46:12.813 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm rm-cluster --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T08:46:19.235 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:19 smithi134 systemd[1]: Stopping Ceph iscsi.iscsi.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:46:19.236 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:19 smithi134 bash[45731]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-iscsi.iscsi.a 2024-02-10T08:46:19.526 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:19 smithi134 bash[39502]: debug Shutdown received 2024-02-10T08:46:30.143 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:29 smithi134 bash[45767]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-iscsi-iscsi-a 2024-02-10T08:46:30.144 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:30 smithi134 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-10T08:46:30.144 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:30 smithi134 bash[45834]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-iscsi-iscsi-a 2024-02-10T08:46:30.627 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:30 smithi134 bash[45844]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-iscsi-iscsi-a-tcmu 2024-02-10T08:46:30.862 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:30 smithi134 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-10T08:46:30.863 INFO:journalctl@ceph.iscsi.iscsi.a.smithi134.stdout:Feb 10 08:46:30 smithi134 systemd[1]: Stopped Ceph iscsi.iscsi.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:46:31.027 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:30 smithi134 systemd[1]: Stopping Ceph node-exporter.b for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:46:31.027 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:30 smithi134 bash[45911]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-node-exporter.b 2024-02-10T08:46:32.026 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:31 smithi134 bash[45945]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-node-exporter-b 2024-02-10T08:46:32.026 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:31 smithi134 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-10T08:46:32.026 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:31 smithi134 bash[46008]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-node-exporter-b 2024-02-10T08:46:32.026 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:31 smithi134 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-10T08:46:32.026 INFO:journalctl@ceph.node-exporter.b.smithi134.stdout:Feb 10 08:46:31 smithi134 systemd[1]: Stopped Ceph node-exporter.b for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:46:32.399 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:32 smithi134 systemd[1]: Stopping Ceph grafana.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715... 2024-02-10T08:46:32.400 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:32 smithi134 bash[46070]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-grafana.a 2024-02-10T08:46:32.776 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:32 smithi134 bash[41763]: t=2024-02-10T08:46:32+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-02-10T08:46:33.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:32 smithi134 bash[46106]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-grafana-a 2024-02-10T08:46:33.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:33 smithi134 bash[46166]: Error response from daemon: No such container: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715-grafana-a 2024-02-10T08:46:33.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:33 smithi134 systemd[1]: ceph-2dd26d0c-c7ec-11ee-95b8-87774f69a715@grafana.a.service: Succeeded. 2024-02-10T08:46:33.276 INFO:journalctl@ceph.grafana.a.smithi134.stdout:Feb 10 08:46:33 smithi134 systemd[1]: Stopped Ceph grafana.a for 2dd26d0c-c7ec-11ee-95b8-87774f69a715. 2024-02-10T08:46:33.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:46:33.894 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T08:46:33.910 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-10T08:46:33.912 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/remote/ubuntu@smithi067.front.sepia.ceph.com/crash 2024-02-10T08:46:33.913 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/crash -- . 2024-02-10T08:46:33.953 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T08:46:33.953 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-02-10T08:46:33.954 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/remote/ubuntu@smithi134.front.sepia.ceph.com/crash 2024-02-10T08:46:33.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/crash -- . 2024-02-10T08:46:33.971 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T08:46:33.972 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-02-10T08:46:33.973 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-10T08:46:33.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | head -n 1 2024-02-10T08:46:34.022 INFO:teuthology.orchestra.run.smithi067.stdout:2024-02-10T08:28:40.551937+0000 mon.a (mon.0) 852 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-10T08:46:34.023 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-10T08:46:34.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[SEC\]' /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | head -n 1 2024-02-10T08:46:34.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]' /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | head -n 1 2024-02-10T08:46:34.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[WRN\]' /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | head -n 1 2024-02-10T08:46:34.212 INFO:teuthology.orchestra.run.smithi067.stdout:2024-02-10T08:28:40.551937+0000 mon.a (mon.0) 852 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-10T08:46:34.213 INFO:tasks.cephadm:Compressing logs... 2024-02-10T08:46:34.214 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T08:46:34.260 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T08:46:34.272 INFO:teuthology.orchestra.run.smithi067.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-02-10T08:46:34.275 INFO:teuthology.orchestra.run.smithi134.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-02-10T08:46:34.286 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.a.log 2024-02-10T08:46:34.286 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.b.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.5.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mgr.x.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mgr.y.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.c.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log: /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T08:46:34.288 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.1.log 2024-02-10T08:46:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log 2024-02-10T08:46:34.290 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T08:46:34.291 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.5.log: /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.6.log 2024-02-10T08:46:34.291 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/tcmu-runner.log 2024-02-10T08:46:34.291 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log 2024-02-10T08:46:34.292 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.1.log: /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.0.log 2024-02-10T08:46:34.293 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.6.log: gzip /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/tcmu-runner.log: -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T08:46:34.293 INFO:teuthology.orchestra.run.smithi134.stderr: 68.0% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/tcmu-runner.log.gz 2024-02-10T08:46:34.293 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.4.log 2024-02-10T08:46:34.293 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log: /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T08:46:34.293 INFO:teuthology.orchestra.run.smithi134.stderr: 81.6%/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.4.log: -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T08:46:34.294 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.7.log 2024-02-10T08:46:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.cephadm.log: 89.0% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T08:46:34.294 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.3.log 2024-02-10T08:46:34.299 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-volume.log: 87.5% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T08:46:34.299 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T08:46:34.300 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.7.log: 92.4% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mgr.x.log.gz 2024-02-10T08:46:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.0.log: 93.0%gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T08:46:34.301 INFO:teuthology.orchestra.run.smithi067.stderr: -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T08:46:34.303 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/cephadm.log: 91.3% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T08:46:34.304 INFO:teuthology.orchestra.run.smithi134.stderr: 92.5% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T08:46:34.307 INFO:teuthology.orchestra.run.smithi134.stderr: 88.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T08:46:34.310 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.2.log 2024-02-10T08:46:34.315 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-volume.log: 95.3% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T08:46:34.320 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T08:46:34.330 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.2.log: 91.3% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T08:46:34.344 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/cephadm.log: 93.7% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-10T08:46:34.350 INFO:teuthology.orchestra.run.smithi067.stderr: 91.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T08:46:34.738 INFO:teuthology.orchestra.run.smithi067.stderr: 91.2% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mgr.y.log.gz 2024-02-10T08:46:35.560 INFO:teuthology.orchestra.run.smithi134.stderr: 93.0% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.b.log.gz 2024-02-10T08:46:35.732 INFO:teuthology.orchestra.run.smithi067.stderr: 92.6% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.c.log.gz 2024-02-10T08:46:36.559 INFO:teuthology.orchestra.run.smithi067.stderr: 92.1% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-mon.a.log.gz 2024-02-10T08:46:38.706 INFO:teuthology.orchestra.run.smithi134.stderr: 94.0% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.6.log.gz 2024-02-10T08:46:39.220 INFO:teuthology.orchestra.run.smithi067.stderr: 94.1% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.0.log.gz 2024-02-10T08:46:39.314 INFO:teuthology.orchestra.run.smithi067.stderr: 94.0% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.2.log.gz 2024-02-10T08:46:39.486 INFO:teuthology.orchestra.run.smithi134.stderr: 94.0% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.7.log.gz 2024-02-10T08:46:39.737 INFO:teuthology.orchestra.run.smithi067.stderr: 94.1% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.3.log.gz 2024-02-10T08:46:39.847 INFO:teuthology.orchestra.run.smithi134.stderr: 94.1% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.5.log.gz 2024-02-10T08:46:39.867 INFO:teuthology.orchestra.run.smithi134.stderr: 94.2% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.4.log.gz 2024-02-10T08:46:39.869 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-02-10T08:46:39.869 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m5.603s 2024-02-10T08:46:39.869 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m21.257s 2024-02-10T08:46:39.869 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.812s 2024-02-10T08:46:40.268 INFO:teuthology.orchestra.run.smithi067.stderr: 94.1% -- replaced with /var/log/ceph/2dd26d0c-c7ec-11ee-95b8-87774f69a715/ceph-osd.1.log.gz 2024-02-10T08:46:40.270 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-10T08:46:40.270 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m6.007s 2024-02-10T08:46:40.270 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m24.819s 2024-02-10T08:46:40.270 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m0.803s 2024-02-10T08:46:40.271 INFO:tasks.cephadm:Archiving logs... 2024-02-10T08:46:40.271 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/remote/ubuntu@smithi067.front.sepia.ceph.com/log 2024-02-10T08:46:40.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T08:46:43.547 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/remote/ubuntu@smithi134.front.sepia.ceph.com/log 2024-02-10T08:46:43.548 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T08:46:46.621 INFO:tasks.cephadm:Removing cluster... 2024-02-10T08:46:46.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo cephadm rm-cluster --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 --force 2024-02-10T08:46:48.220 DEBUG:teuthology.orchestra.run.smithi134:> sudo cephadm rm-cluster --fsid 2dd26d0c-c7ec-11ee-95b8-87774f69a715 --force 2024-02-10T08:46:49.828 INFO:tasks.cephadm:Teardown complete 2024-02-10T08:46:49.829 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-10T08:46:49.838 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-10T08:46:49.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-10T08:46:49.841 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-10T08:46:49.903 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-10T08:46:49.915 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-10T08:46:49.915 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T08:46:49.918 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T08:46:49.938 INFO:teuthology.orchestra.run.smithi067.stdout: remote refid st t when poll reach delay offset jitter 2024-02-10T08:46:49.938 INFO:teuthology.orchestra.run.smithi067.stdout:============================================================================== 2024-02-10T08:46:49.938 INFO:teuthology.orchestra.run.smithi067.stdout:*hv01.front.sepi 44.4.53.2 2 u 107 128 377 0.094 0.006 0.058 2024-02-10T08:46:49.938 INFO:teuthology.orchestra.run.smithi067.stdout:+hv02.front.sepi 96.245.170.99 3 u 55 128 377 0.093 0.068 0.091 2024-02-10T08:46:49.938 INFO:teuthology.orchestra.run.smithi067.stdout:+hv03.front.sepi 74.6.168.72 3 u 42 128 377 0.101 0.535 0.096 2024-02-10T08:46:49.938 INFO:teuthology.orchestra.run.smithi067.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-02-10T08:46:49.939 INFO:teuthology.orchestra.run.smithi134.stdout: remote refid st t when poll reach delay offset jitter 2024-02-10T08:46:49.940 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================== 2024-02-10T08:46:49.940 INFO:teuthology.orchestra.run.smithi134.stdout:*hv01.front.sepi 44.4.53.2 2 u 58 128 377 0.087 -0.690 0.060 2024-02-10T08:46:49.940 INFO:teuthology.orchestra.run.smithi134.stdout:+hv02.front.sepi 96.245.170.99 3 u 40 128 377 0.090 -0.571 0.042 2024-02-10T08:46:49.940 INFO:teuthology.orchestra.run.smithi134.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 128 377 0.098 -0.106 0.056 2024-02-10T08:46:49.940 INFO:teuthology.orchestra.run.smithi134.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-02-10T08:46:49.940 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-10T08:46:49.950 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-10T08:46:49.951 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-10T08:46:49.960 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-10T08:46:49.969 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-10T08:46:49.978 INFO:teuthology.task.internal:Duration was 2488.051826 seconds 2024-02-10T08:46:49.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-10T08:46:49.987 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-10T08:46:49.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T08:46:49.990 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T08:46:50.035 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-10T08:46:50.035 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-02-10T08:46:50.036 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T08:46:50.092 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-02-10T08:46:50.092 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T08:46:50.107 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-10T08:46:50.107 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T08:46:50.136 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T08:46:50.214 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-10T08:46:50.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T08:46:50.216 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T08:46:50.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-10T08:46:50.406 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-10T08:46:50.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T08:46:50.423 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T08:46:50.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-10T08:46:50.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:46:50.472 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:46:50.483 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-02-10T08:46:50.496 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-02-10T08:46:50.519 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:46:50.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:46:50.544 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T08:46:50.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T08:46:50.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-10T08:46:50.580 INFO:teuthology.task.internal:Transferring archived files... 2024-02-10T08:46:50.581 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/remote/smithi067 2024-02-10T08:46:50.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T08:46:50.622 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298/remote/smithi134 2024-02-10T08:46:50.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T08:46:50.653 INFO:teuthology.task.internal:Removing archive directory... 2024-02-10T08:46:50.653 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T08:46:50.659 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T08:46:50.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-10T08:46:50.708 INFO:teuthology.task.internal:Not uploading archives. 2024-02-10T08:46:50.708 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-10T08:46:50.718 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-10T08:46:50.719 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T08:46:50.721 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T08:46:50.727 INFO:teuthology.orchestra.run.smithi067.stdout: 1048617 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 08:46 /home/ubuntu/cephtest 2024-02-10T08:46:50.745 INFO:teuthology.orchestra.run.smithi134.stdout: 1048601 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 08:46 /home/ubuntu/cephtest 2024-02-10T08:46:50.747 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-10T08:46:50.757 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-10T08:46:50.776 INFO:teuthology.nuke:Checking targets against current locks 2024-02-10T08:46:50.802 DEBUG:teuthology.nuke:shortname: smithi067 2024-02-10T08:46:50.802 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T08:46:50.824 DEBUG:teuthology.nuke:shortname: smithi134 2024-02-10T08:46:50.824 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T08:46:50.830 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:54:47.659329', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-02-10T08:46:50.846 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_21:34:44-rados-wip-yuri10-testing-2024-02-08-0854-pacific-distro-default-smithi/7555298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:54:47.660806', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2024-02-10T08:46:50.877 INFO:teuthology.orchestra.console.smithi067:Power off 2024-02-10T08:46:50.878 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T08:46:50.901 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2024-02-10T08:46:50.901 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:46:50.917 INFO:teuthology.orchestra.console.smithi134:Power off 2024-02-10T08:46:50.917 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T08:46:50.927 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-10T08:46:50.941 DEBUG:teuthology.orchestra.console.smithi134:power off output: Chassis Power Control: Down/Off 2024-02-10T08:46:50.941 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:46:50.966 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-02-10T08:46:54.929 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:46:54.967 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:46:55.040 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-10T08:46:55.078 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is on 2024-02-10T08:46:59.044 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:46:59.079 DEBUG:teuthology.orchestra.console.smithi134:pexpect command: ipmitool -H smithi134.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T08:46:59.155 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2024-02-10T08:46:59.188 DEBUG:teuthology.orchestra.console.smithi134:check power output: Chassis Power is off 2024-02-10T08:46:59.257 INFO:teuthology.orchestra.console.smithi067:Power off completed 2024-02-10T08:46:59.289 INFO:teuthology.orchestra.console.smithi134:Power off completed 2024-02-10T08:46:59.395 INFO:teuthology.run:Summary data: description: rados/cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} duration: 2488.051826477051 failure_reason: '"2024-02-10T08:28:40.551937+0000 mon.a (mon.0) 852 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-02-10T08:46:59.396 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T08:46:59.486 INFO:teuthology.run:FAIL